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Abstract 

In this paper I am presenting an overview on several topics related to nonequilibrium 
fluctuations in small systems. I start with a general discussion about fluctuation theorems 
and applications to physical examples extracted from physics and biology: a bead in an 
optical trap and single molecule force experiments. Next I present a general discussion 
on path thermodynamics and consider distributions of work/heat fluctuations as large 
deviation functions. Then I address the topic of glassy dynamics from the perspective 
of nonequilibrium fluctuations due to small cooperatively rearranging regions. Finally, I 
conclude with a brief digression on future perspectives. 
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1 What are small systems? 

Thermodynamics, a scientific discipline inherited from the 18th century, is facing new chal- 
lenges in the description of nonequilibrium small (sometimes also called mesoscopic) systems. 
Thermodynamics is a discipline built in order to explain and interpret energetic processes 
occurring in macroscopic systems made out of a large number of molecules on the order of the 
Avogadro number. Although thermodynamics makes general statements beyond reversible 
processes its full applicability is found in equilibrium systems where it can make quantitative 
predictions just based on a few laws. The subsequent development of statistical mechanics has 
provided a solid probabilistic basis to thermodynamics and increased its predictive power at 
the same time. The development of statistical mechanics goes together with the establishment 
of the molecular hypothesis. Matter is made out of interacting molecules in motion. Heat, 
energy and work are measurable quantities that depend on the motion of molecules. The laws 
of thermodynamics operate at all scales. 

Let us now consider the case of heat conduction along polymer fibers. Thermodynam- 
ics applies at the microscopic or molecular scale, where heat conduction takes place along 
molecules linked along a single polymer fiber, up to the macroscopic scale where heat is trans- 
mitted through all the fibers that make a piece of rubber. The main difference between the 
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two cases is the amount of heat transmitted along the system per unit of time. In the first 
case the amount of heat can be a few ksT per mihisecond whereas in the second can be on 
the order of NfksT where Nf is the number of polymer fibers in the piece of rubber. The 
relative amplitude of the heat fluctuations are on the order of 1 in the molecular case and 
1/ ^Nf in the macroscopic case. Because Nj is usually very large, the relative magnitude of 
heat fluctuations is negligible for the piece of rubber as compared to the single polymer fiber. 
We then say that the single polymer fiber is a small system whereas the piece of rubber is a 
macroscopic system made out of a very large collection of small systems that are assembled 
together. 

Small systems are those in which the energy exchanged with the environment is a few times 
and energy fluctuations are observable. A few can be 10 or 1000 depending on the system. 
A small system must not necessarily be of molecular size or contain a few number of molecules. 
For example, a single polymer chain may behave as a small system although it contains millions 
of covalently linked monomer units. At the same time, a molecular system may not be small 
if the transferred energy is measured over long times compared to the characteristic heat 
diffusion time. In that case the average energy exchanged with the environment during a time 
interval t can be as large as desired by choosing t large enough. Conversely, a macroscopic 
system operating at short time scales could deliver a tiny amount of energy to the environment, 
small enough for fluctuations to be observable and the system being effectively small. 

Because macroscopic systems are collections of many molecules we expect that the same 
laws that have been found to be applicable in macroscopic systems are also valid in small 
systems containing a few number of molecules [HE]. Yet, the phenomena that we will observe 
in the two regimes will be different. Fluctuations in large systems are mostly determined by 
the conditions of the environment. Large deviations from the average behavior are hardly 
observable and the structural properties of the system cannot be inferred from the spectrum 
of fluctuations. In contrast, small systems will display large deviations from their average 
behavior. These turn out to be less sensitive to the conditions of the surrounding environment 
(temperature, pressure, chemical potential) and carry information about the structure of the 
system and its nonequilibrium behavior. We may then say that information about the structure 
is carried in the tails of the statistical distributions describing molecular properties. 

The world surrounding us is mostly out of equilibrium, equilibrium being just an idealiza- 
tion that requires of specific conditions to be met in the laboratory. Even today we do not 
have a general theory about nonequilibrium macroscopic systems as we have for equilibrium 
ones. Onsager theory is probably the most successful attempt albeit its domain of validity 
is restricted to the linear response regime. In small systems the situation seems to be the 
opposite. Over the past years, a set of theoretical results, that go under the name of fluctu- 
ation theorems, have been unveiled . These theorems make specific predictions about energy 
processes in small systems that can be scrutinized in the laboratory. 

The interest of the scientific community on small systems has been boosted by the recent 
advent of micromanipulation techniques and nanotechnologies. These provide adequate sci- 
entific instruments that can measure tiny energies in physical systems under nonequilibrium 
conditions. Most of the excitement comes also from the more or less recent observation that 
biological matter has successfully exploited the smallness of biomolecular structures (such as 
complexes made out of nucleic acids and proteins) and the fact that they are embedded in 
nonequilibrium environment to become wonderfully complex and efficient at the same time 

[as]. 
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The goal of this review is to discuss these ideas from a physicsist perspective by emphasizing 
the underlying common aspects in a broad category of systems, from glasses to biomolecules. 
We aim to put together some concepts in statistical mechanics that may become the building 
blocks underlying a future theory of nonequilibrium small systems. This is not a review in 
the traditional sense but rather a survey of a few selected topics in nonequilibrium statistical 
mechanics concerning systems that range from physics to biology. The selection is biased 
by my own particular taste and expertise. For this reason I have not tried to cover most of 
the relevant references for each selected topic but rather emphasize a few of them that make 
explicit connection with my discourse. Interested readers are advised to look at other reviews 
that have been recently written on related subjects O [H [7] 

The outline of the review is as follows. Section [2] introduces two examples, one from 
physics and the other from biology, that are paradigms of nonequilibrium behavior. Section [3] 
is devoted to cover most important aspects of fluctuation theorems whereas Section H] presents 
applications of fluctuation theorems to physics and biology. Section [5] presents the discipline 
of path thermodynamics and briefly discusses large deviation functions. Section [H] discusses 
the topic of glassy dynamics from the perspective of nonequilibrium fluctuations in small 
cooperatively rearranging regions. We conclude with a brief discussion on future perspectives. 

2 Small systems in physics and biology 
2.1 Colloidal systems 

Condensed matter physics is full of examples where nonequilibrium fluctuations of mesoscopic 
regions governs the nonequilibrium behavior that is observed at the macroscopic level. A 
class of systems that have attracted a lot of attention for many decades and that still remain 
poorly understood are glassy systems, such as supercooled liquids and soft materials [8]. Glassy 
systems can be prepared in a nonequilibrium state, e.g. by fast quenching the sample from 
high to low temperatures, and subsequently following the time evolution of the system as a 
function of time (also called the "age" of the system) . Glassy systems display extremely slow 
relaxation and aging behavior, i.e. an age dependent response to the action of an external 
perturbation. Aging systems respond slower as they get older keeping memory of their age 
for timescales that range from picoseconds to years. The slow dynamics observed in glassy 
systems is dominated by intermittent, large and rare fluctuations where mesoscopic regions 
release some stress energy to the environment. Current experimental evidence suggests that 
these events correspond to structural rearrangements of clusters of molecules inside the glass 
that release some energy through an activated and cooperative process. These cooperatively 
rearranging regions are responsible of the heterogeneous dynamics observed in glassy systems 
and lead to a great disparity of relaxation times. The fact that, under appropriate conditions, 
the slow relaxation observed in glassy systems virtually takes forever, indicates that the average 
amount of energy released in any rearrangement event must be small enough to account for 
an overall net energy release of the whole sample (which is not larger than the stress energy 
contained in the system in the initial nonequilibrium state). 

In some systems, such as colloids, the free-volume (i.e. the volume of the system that is 
available for motion to the colloidal particles) is the relevant variable, and the volume fraction 
of colloidal particles (/> is the parameter governing the relaxation rate. Relaxation in colloidal 
systems is determined by the release of tensional stress energy and free volume in spatial re- 
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gions that contain a few particles. Colloidal systems offer great advantages to do experiments 
for several reasons: 1) In colloids the control parameter is the volume fraction, 0, a quantity 
easy to control in experiments; 2) Under appropriate solvent conditions colloidal particles 
behave as hard spheres, a system that is pretty well known and has been theoretically and nu- 
merically studied for many years; 3) The size of colloidal particles is typically of a few microns 
making possible to follow the motion of a small number particles using video microscopy and 
spectroscopic techniques. This allows to detect cooperatively rearranging clusters of particles 
and characterize their heterogeneous dynamics. Experiments have been done with PMMA 
poly(methyl methacrylate) particles of ~ 1/im radius suspended in organic solvents [H [TO] . 
Confocal microscopy then allows to acquire images of spatial regions of extension on the or- 
der of tens of microns that contain a few thousand of particles, small enough to detect the 
collective motion of clusters. In experiments carried by Weeks and collaborators [llj a highly 
stressed nonequilibrium state is produced by mechanically stirring a colloidal system at vol- 
ume fractions (j) ^ (pg where (j)g is the value of the volume fraction at the glass transition where 
colloidal motion arrests. The subsequent motion is then observed. A few experimental results 
are shown in Figure [H The mean square displacement of the particles inside the confocal 
region show aging behavior. Importantly also, the region observed is small enough to observe 
temporal heterogeneity, i.e. the aging behavior is not smooth with the age of the system as 
usually observed in light scattering experiments. Finally, the mean square displacement for a 
single trajectory shows abrupt events characteristic of collective motions involving a few tens 
of particles. By analyzing the average number of particles belonging to a single cluster Weeks 
et al. [ll] find that no more than 40 particles participate in the rearrangment of a single 
cluster suggesting that cooperatively rearranging regions are not larger than a few particle 
radii in extension. Large deviations, intermittent events and heterogeneous kinetics are the 
main features observed in these experiments. 

2.2 Molecular machines 

Biochemistry and molecular biology are scientific disciplines aiming to describe the structure, 
organization and function of living matter [12^ I13j. Both disciplines seek an understanding 
of life processes in molecular terms. Their main object of study are biological molecules 
and the function they play in the biological process where they intervene. Biomolecules are 
small systems from several points of view. First, from their size, where they span just a few 
nanometers of extension. Second, from the energies they require to function properly, which 
is determined by the amount of energy that can be extracted by hydrolyzing one molecule 
of ATP (approximately 12 fc^T at room temperature or 300/C). Third, from the typically 
short amount of time that it takes to complete an intermediate step in a biological reaction. 
Inside the cell many reactions that would take an enormous long time under non-biological 
conditions are speeded up by several orders of magnitude in the presence of specific enzymes. 

Molecular machines (also called molecular motors) are amazing complexes made out of 
several parts or domains that coordinate their behavior to perform specific biological functions 
by operating out of equilibrium. Molecular machines hydrolyze energy carrier molecules such 
as ATP to transform the chemical energy contained in the high energy bonds into mechanical 
motion |14t [TCI [TBI E] • An example of a molecular machine that has been studied by molecular 
biologists and biophysicists is the RNA polymerase [181 [19] . This is an enzyme that synthesizes 
the pre-messenger RNA molecule by translocating along the DNA and reading, step by step. 
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Figure 1: (Left inset) A snapshot picture of a colloidal system obtained with confocal mi- 
croscopy. (Left) Aging behavior observed in the mean square displacement (MSD), (Ax^), as 
a function of time for different ages. The colloidal system reorganizes slower as it becomes 
older. (Right) 7 = \/ {Ax^) /3 (upper curve) and (Ax^) (lower curve) as a function of the age 
measured over a fixed time window At = lOmin. For a diffusive dynamics both curves should 
coincide, however these measurements show deviations from diffusive dynamics as well as in- 
termittent behavior (Inset: the same as in main panel but plotted in logarithmic timescale). 
Figures (A,B) taken from http://www.physics.emory.edu/weeks and Figure (C) taken from 
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the sequence of bases along the DNA backbone. The read out of the RNA polymerase is 
exported from the nucleus to the cytoplasm of the cell to later be translated in the ribosome, 
a huge molecular machine that synthesizes the protein coded into the messenger RNA [20] . 
Using single molecule experiments it is possible to grab one DNA molecule by both ends using 
optical tweezers and follow the translocation motion of the RNA polymerase [2H [22] . Current 
optical tweezer techniques have even resolved the motion of the enzyme at the level of a single 
base pair [23[l24j. The experiment requires flowing inside the fluidics chamber the enzymes and 
proteins that are necessary to initiate the transcription reaction. The subsequent motion and 
transcription by the RNA polymerase is called elongation and can be studied under applied 
force conditions that assist or oppose the motion of the enzyme [25] . In Figure [2] we show 
the results obtained in the Bustamante group for the RNA polymerase of Escherichia Coli, 
a bacteria found in the intestinal tracts of animals. In Figure [2j^ the polymerase apparently 
moves at a constant average speed but is characterized by pauses (black arrows) where motion 
temporarily arrests. In Figure [2f3 we show the transcription rate (or speed of the enzyme) as 
a function of time. Note the large intermittent fluctuations in the transcription rate, a typical 
feature of small systems embedded in a noisy thermal environment. In contrast to the slow 
aging dynamics observed in colloidal systems (Sec. 12. ip the kinetic motion of the polymerase 
is steady and fast (translocation rates are typically on the order of few tens of base pairs 
per second). We then say that the polymerase is in a nonequilibrium steady state. As in the 
previous case, large deviations, intermittent events (e.g. temporary pauses in the translocation 
motion) and complex kinetics are the main features observed in these experiments. 

3 Fluctuation theorems (FTs) 

Fluctuation theorems (FTs) make statements about energy exchanges that take place between 
a system and its surroundings under general nonequilibrium conditions. Since their discovery 
in the mid 90's [261 [271 [28] there has been a growing interest to elucidate their importance 
and implications. FTs provide a fresh look to our understanding of old questions such as the 
origin of irreversibility and the second law in statistical mechanics [291 [3t)j . In addition, FTs 
provide statements about energy fluctuations in small systems which, under generic conditions, 
should be experimentally observable. FTs have been discussed in the context of deterministic, 
stochastic and thermostatted systems. Although the results obtained differ depending on the 
particular model of the dynamics that is used, in the nutshell they look pretty equivalent. 

FTs are related to the so called nonequilibrium work relations introduced by Jarzynski 
[5T] . This fundamental relation can be seen as consequence of the FTs [321 E3] • It represents a 
new result beyond classical thermodynamics that shows the possibility to recover free energy 
differences using irreversible processes. Several reviews have been written on the subject 
[HI (Ml (Ml [371 13 with specific emphasis on theory and/or experiments. In the next sections 
we review some of the main results. Throughout the text we will take ks = 1- 

3.1 Nonequilibrium states 

An important concept in thermodynamics is the state variable. State variables are those that, 
once determined, uniquely specify the thermodynamic state of the system. Examples are the 
temperature, the pressure, the volume and the mass of the different components in a given 
system. To specify the state variables of a system it is common to put the system in contact 
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Figure 2: (Left) Experimental setup in force-flow measurements. Optical tweezers are used to 
trap beads but forces are applied on the RNApol-DNA molecular complex using the Stokes drag 
force acting on the left bead immersed in the flow. In this setup force assists RNA transcription 
as the DNA tether between beads increases in length as a function of time. (A) The contour 
length of the DNA tether as a function of time (blue curve) and (B) the transcription rate 
(red curve) as a function of the contour length. Pauses (temporary arrests of transcription) 
are shown as vertical arrows. Figure taken from 
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with a bath. The bath is any set of sources (of energy, volume, mass, etc.) large enough to 
remain unaffected by the interaction with the system under study. The bath ensures that 
a system can reach a given temperature, pressure, volume and mass concentrations of the 
different components when put in thermal contact with the bath (i.e. with all the relevant 
sources) . Equilibrium states are then generated by putting the system in contact with a bath 
and waiting until the system properties relax to the equilibrium values. Under such conditions 
the system properties do not change with time and the average net heat /work/mass exchanged 
between the system and the bath is zero. 

Nonequilibrium states can be produced under a great variety of conditions, either by 
continuously changing the parameters of the bath or by preparing the system in an initial 
nonequilibrium state that slowly relaxes toward equilibrium. In general a nonequilibrium state 
is produced whenever the system properties change with time and/or the net heat /work/mass 
exchanged by the system and the bath is non zero. We can distinguish at least three different 
types of nonequilibrium states: 

• Nonequilibrium transient state (NETS). The system is initially prepared in an 
equilibrium state and later driven out of equilibrium by switching on an external per- 
turbation. The system quickly returns to a new equilibrium state once the external 
perturbation stops changing. 

• Nonequilibrium steady-state (NESS). The system is driven by external forces (ei- 
ther time dependent or non-conservative) in a stationary nonequilibrium state where its 
properties do not change with time. The steady state is an irreversible nonequilibrium 
process, that cannot be described by the Boltzmann-Gibbs distribution, where the aver- 
age heat that is dissipated by the system (equal to the entropy production of the bath) 
is positive. 

There are still other categories of NESS. For example in nonequilibrium transient steady 
states the system starts in a nonequilibrium steady state but is driven out of equilibrium 
by an external perturbation to finally settle in a new nonequilibrium steady state. 

• Nonequilibrium aging state (NEAS). The system is initially prepared in a nonequi- 
librium state and put in contact with the sources. The system is then let evolve alone 
but fails to reach thermal equilibrium in observable or laboratory time scales. In this 
case the system is in a non-stationary slowly relaxing nonequilibrium state called aging 
state and characterized by a very small heat dissipation to the sources. In the aging 
state two-times correlations decay slower as the system becomes older. Two-time cor- 
relation functions depend on both times and not just on the difference of times (time 
translational invariance is said to be broken). 

There are many examples of nonequilibrium states. A classic example of a NESS is an 
electrical circuit made out of a battery and a resistance. The current flows through the 
resistance and the chemical energy stored in the battery is dissipated to the environment in 
the form of heat; the average dissipated power, V^is = VI, is identical to the power supplied 
by the battery. Another example is a sheared fluid between two plates or coverslips where 
one of them is moved relative to the other at a constant velocity v. To sustain such state a 
mechanical power that is equal to V oc rjv^ has to be exerted upon the moving plate where 
•q is the shear viscosity of the fluid. The mechanical work produced is then dissipated in the 
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form of heat through the viscous friction between contiguous fluid layers. Another examples 
of NESS are chemical reactions in metabolic pathways that are sustained by activated carrier 
molecules such as ATP. In such case, hydrolysis of ATP is strongly coupled to specific oxidative 
reactions. For example, ionic channels use ATP hydrolysis to transport protons against the 
electromotive force. 

A classic example of NETS is the case of a protein in its initial native state that is me- 
chanically pulled (e.g. using AFM) by exerting force at the ends of the molecule. The protein 
is initially folded and in thermal equilibrium with the surrounding aqueous solvent. By me- 
chanically stretching the protein is pulled away from equilibrium into a transient state until 
it finally settles into the unfolded and extended new equilibrium state. Another example of 
a NETS is a bead immersed in water and trapped in an optical well generated by a focused 
laser beam. When the trap is moved to a new position (e.g. by redirecting the laser beams) 
the bead is driven into a NETS. After some time the bead reaches again equilibrium at the 
new position of the trap. In another experiment the trap is suddenly put in motion at a speed 
V so the bead is transiently driven away from its equilibrium average position until it settles 
into a NESS characterized by the speed of the trap. This results in the average position of 
the bead lagging behind the position of the center of the trap. 

The classic example of a NEAS is a supercooled liquid cooled below its glass transition 
temperature. The liquid solidifies into an amorphous slowly relaxing state characterized by 
huge relaxational times and anomalous low frequency response. Other systems are colloids 
that can be prepared in a NEAS by the sudden reduction of the volume fraction of the colloidal 
particles or by putting the system under a strain/stress. 

The classes of nonequilibrium states previously described do not make distinctions whether 
the system is macroscopic or small. In small systems, however, it is common to speak about 
the control parameter to emphasize the importance of the constraints imposed by the bath 
that are externally controlled and do not fluctuate. The control parameter (A) represents a 
value (in general, a set of values) that defines the state of the bath. Its value determines the 
equilibrium properties of the system, e.g. the equation of state. In macroscopic systems it is 
unnecessary to discern which value is externally controlled because fluctuations are small and 
all equilibrium ensembles give the same equivalent thermodynamic description, i.e. the same 
equation of state. Differences arise only when including fluctuations in the description. The 
nonequilibrium behavior of small systems is then strongly dependent on the protocol used 
to drive them out of equilibrium. The protocol is generally defined by the time evolution of 
the control parameter X{t). As a consequence, the characterization of the protocol X{t) is 
an essential step to unambiguously define how the nonequilibrium state has been generated. 
Figure [3] shows a representation of a few examples of NESS and control parameters. 

3.2 Fluctuation theorems (FTs) in stochastic dynamics 

In this section we are presenting a derivation of the FT based on stochastic dynamics. In con- 
trast to deterministic systems, stochastic dynamics is naturally endowed of crucial assumptions 
that are needed for the derivation such as the ergodicity hypothesis. The derivation we are 
presenting here follows the approach introduced by Crooks-Kurchan-Lebowitz-Spohn [381 139j 
and includes also some results recently obtained by Seifert using Langevin systems [10] . 
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Figure 3: Examples of NESS. (A) An electric current I flowing through a resistance R and 
maintained by a voltage source or control parameter V. (B) A fluid sheared between two plates 
moving relative to each other at speed v (the control parameter) . (C) A chemical reaction 
A ^ B coupled to ATP hydrolysis. The control parameter here is relative concentration of 
ATP and ADP. 

3.2.1 The master equation 

Let us consider a stochastic system described by a generic variable C. This variable may stand 
for the position of a bead in an optical trap, the velocity field of a fluid, the current passing 
through a resistance, the number of native contacts in a protein, etc. A trajectory or path T 
in configurational space is described by a discrete sequence of configurations in phase space, 



where the system occupies configuration Ck at time t^ = kAt and At is the duration of the 
discretized elementary time-step. In what follows we consider paths that start at Cq at time 
t = and end at the configuration Cm at time t = MAt. The continuous time limit is 
recovered by taking M — oo, At — for a fixed value of t. 

Let ((...)) denote the average over all paths that start at t = at configurations Cq initially 
chosen from a distribution Po{C). We also define Pk{C) as the probability, measured over all 
possible dynamical paths, that the system is in configuration C at time t^ = kAt. Probabilities 
are normalized for any k, 



The system is assumed to be in contact with a thermal bath at temperature T. We also 
assume that the microscopic dynamics of the system is of the Markovian type: the probability 

for the system to be at a given configuration and at a given time only depends on its previous 
configuration. We then introduce the transition probability Wk{C — > C). This denotes the 
probability for the system to change from C to C at time-step k. According to Bayes formula. 



r = {Co,Ci,C2, ...,Cm} 



(1) 




(2) 



c 




(3) 



c 
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where the W's satisfy the normahzation condition, 



^W,(C-C') = 1 . (4) 
c 

Using ([2]), ([3]) we can write the following Master equation for the probability Pk{C), 

APkiC) = Pk+i{C) - PkiC) = WkiC ^ C)PkiC') -Y^miC^ C')Pk{C) (5) 

where the terms C = C have not been included as they cancel out in the first and second sums 
in the r.h.s. The first term in the r.h.s accounts for all transitions leading to the configuration 
C whereas the second term counts all processes leaving C. It is convenient to introduce the 
rates rt{C C) in the continuous time limit At — > 0, 

rt{C ^ C) = lim ^k{C -^C') , 



Equation @ becomes, 

dPt(C) 



Y: n{C' - C)Pt{C') - Y rr{C - C')PtiC) . (7) 



dt 



3.2.2 Microscopic reversibility 

We now introduce the concept of the control parameter A (see Sec. 13. In the present scheme 
the discrete-time sequence {A^; < A; < M} defines the perturbation protocol. The transition 
probability Wfc(C — > C) now depends explicitly on time through the value of an external time- 
dependent parameter A^. The parameter Afc may indicate any sort of externally controlled 
variable that determines the state of the system, for instance the value of the external magnetic 
field applied on a magnetic system, the value of the mechanical force applied to the ends of 
a molecule, the position of a piston containing a gas or the concentrations of ATP and ADP 
in a molecular reaction coupled to hydrolisis (see Figure ^ . The time variation of the control 
parameter, A = {Xk+i ~ Xk)/^ti is used as tunable parameter which determines how strongly 
irreversible is the nonequilibrium process. In order to emphasize the importance of the control 
parameter, in what follows we will parametrize probabilities and transition probabilities by 
the value of the control parameter at time-step k, A (rather than by the time t). Therefore 
we will write P\(C),Wx(C — > C) for the probabilities and transition probabilities respectively 
at a given time t. 

The transition probabilities W\{C C) cannot be arbitrary but must guarantee that the 
equilibrium state P'^{C) is a stationary solution of the master equation The simplest 
way to impose such condition is to model the microscopic dynamics as ergodic and reversible 
for a fixed value of A, 

mjc^c) ^ pi\c') 

Wx{C'^C) Pl^'iC) ■ ^' 

The latter condition is commonly known as microscopic reversibility or local detailed balance. 
This property is equivalent to time reversal invariance in deterministic (e.g. thermostatted) 
dynamics. Microscopic reversibility, a common assumption in nonequilibrium statistical me- 
chanics, is the crucial ingredient in the present derivation. 
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Equation ([8]) has been criticized as a relation that is vahd only in the vicinity of equilibrium 
because the rates appearing in ([8]) are related to the equilibrium distribution P^'^{C). However, 
we must observe that the equilibrium distribution evaluated at a given configuration depends 
only on the Hamiltonian of the system at that configuration. Therefore, dS]) must be read as 
a relation that only depends on the energy of configurations. It should hold close but also far 
from equilibrium by properly defining the configurational space. 

Let us now consider all possible dynamical paths T that are generated starting from an 
ensemble of initial configurations at time (described by the initial distribution Pxq{C)) and 
which evolve according to ([8|) until time t (t = MAt, M being the total number of discrete 
time-steps). Dynamical evolution takes place according to a given protocol, {Afe,0 < k < 
M}, the protocol defining the nonequilibrium experiment. Different dynamical paths will be 
generated because of the different initial conditions (all weighted with the probability Pxo(C)) 
and because of the stochastic nature of the transitions between configurations at consecutive 
time-steps. 



3.2.3 The nonequilibrium equality 

Let us consider a generic observable A{T). The average value of A is given by, 

(A) = Y: Pir)A{r) (9) 

r 

where F denotes the path and P{T) indicates the probability of that path. Using the fact that 
the dynamics is Markovian together with the definition ([T]) we can write, 

p(r) = Pao (Co) n >^A, (Ck - Ck+i) . (10) 

k=0 

By inserting (llOp into ([9]) we obtain, 

M-l 

(A) = J2AiT)Px,iCo) n WA,(Cfc - Ck+i) . (11) 
r k=o 

Using the detailed balance condition ([H]) this expression reduces to, 

M-i P^^(C ) 

{A) =J2Px,iCo)AiT) n [WA,(Cfe+i - C,)^^] (12) 
r k=o ^Afel.^fcJ 

= ^^(r)P,„(Co)exp[^ log(^l_iH)] Jl Wx,iCk+i^C,) . (13) 

r k=0 Afe'.'-fc/ k=Q 

This equation can not be worked out further. However, let us consider the following 
observable S{T), defined by, 

AiT)^eM-SiT))^^,U{M^) (1^) 
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where 6(C) is any positive definite and normalizable function, 



E KC) = 1 (15) 

c 

and Pao(Co) > 0,VCo. By inserting ^ into ^ we get, 

M-l 

(exp(-5)) = E KCm) n WA,(Cfe+i ^ Cfc) = 1 (16) 
r fc=o 

where we have apphed a telescopic sum (we first summed over Cm, used (fT5]l and subsequently 
summed over the rest of variables by using dH)). We call S(T) the total dissipation of the 
system. It is given by, 

M-l P'^'^iC ) 

S{T) = E l°g[ peir' 1 + log(Ao(Co)) - log(6(CM)) . (17) 



k=0 



The equality ([16]) immediately implies, by using Jensen's inequality, the following inequality, 

{S) > (18) 

which is reminiscent of second law of thermodynamics for nonequilibrium systems: the entropy 
of the universe (system plus the environment) always increases. Yet, we have to identify the 
different terms appearing in (|17p . It is important to stress that entropy production in nonequi- 
librium systems can be defined just in terms of the work/heat /mass transferred by the system 
to the external sources which represent the bath. The definition of the total dissipation ()17p 
is arbitrary because it depends on an undetermined function b{C) (jlSp . Therefore, the total 
dissipation S may not necessarily have a general physical meaning and could be interpreted 
in different ways depending on the specific nonequilibrium context. 

Equation (I16p has appeared in the past in the literature 41j and is mathematically related 
to the Jarzynski equality [3l]. We analyze this connection below in Sec. I3.3.T1 

3.2.4 The fluctuation theorem 

A physical insight on the meaning of the total dissipation S can be obtained by deriving the 
fluctuation theorem. We start by defining the reverse path T* of a given path T. Let us 
consider the path, F = Co — > Ci — > ... — > Cm corresponding to the forward (F) protocol which 
is described by the sequence of values of A at different time-steps k, A^. Every transition 
occurring at time-step k , Ck ^ Ck+i, is governed by the transition probability Wx^,{Ck 
Cjt+i). The reverse path of F is defined as the time reverse sequence of configurations, F* = 
Cm — *■ Cm-1 — > ••• — > Co corresponding to the reverse (R) protocol described by the time- 
reversed sequence of values of A, A|? = XM-k-i- 

The probability of a given path and its reverse are given by, 

A/-1 

VFiT)= n >VA,(C,^Cfc+i) (19) 

k=0 

M-l M-l 

w*) = n WA«(CAf-fe ^ CM-k-i) = n >^A,(Cfc+i ^ Ck) (20) 

A;=0 k=0 
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where in the last hne we have shifted variables k M — 1 — k. We use the notation V for 
the path probabilities rather than the usual letter P. This difference in notation is introduced 
to stress the fact that path probabilities ()19|20|) are non-normalized conditional probabilities, 
i.e. Er^F(i?)(r) / 1. By using ([H]) we get, 

Pfl(r-)"M PIT,) - ^'^Pf*'*'^" '2') 

where we defined the entropy production of the system, 

M-i p'^'ifr \ 

Note that Sp{T) is just a part of the total dissipation introduced in pT|) . 

S{T) = Sp{T) + B{T) (23) 

where B{T) is the so-called boundary term, 

5(r) = log(PAo(Co))-log(6(CM)) . (24) 

We tend to identify ^^(r) as the entropy production in a nonequilibrium system whereas B(T) 
is a term that contributes just at the beginning and end of the nonequilibrium process. Note 
that the entropy production Sp(T) is antisymmetric under time reversal, ^^(r*) = —Sp{r), 
expressing the fact that the entropy production is a quantity related to irreversible motion. 
According to (|2ip paths that produce a given amount of entropy are much more probable 
than those that consume the same amount of entropy. How much improbable is entropy 
consumption depends exponentially on the amount of entropy consumed. The larger the 
system is, the larger the probability to produce (rather than consume) a given amount of 
entropy Sp. 

Equation ()21|] has already the form of a fluctuation theorem. However, in order to get a 
proper fluctuation theorem we still need to specify relations between probabilities for phys- 
ically measurable observables rather than paths. From ()2ip it is straightforward to derive a 
fluctuation theorem for the total dissipation S. Let us take b{C) = Pam(^)- With this choice 
we get, 

S{T) = Sp{T) + BiT) = E log(^%^) + log(PAo(Co)) - log(PA„(CM)) • (25) 

The physical motivation behind this choice is that S now becomes an antisymmetric observable 
under time reversal. Albeit 5'p(r) is always antisymmetric, the choice (I25p is the only one that 
guarantees that the total dissipation S changes sign upon reversal of the path, 5(r*) = —S{T). 
The symmetry property of observables under time reversal and the possibility to consider 
boundary terms where S is symmetric (rather than antisymmetric) under time reversal has 
been discussed in 
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The probability to produce a total dissipation S along the forward protocol is given by, 



Pf{S) = Px,{Co)VF{mS{T) -S)=Y^ Pao {Co)VR{n eMSp{rmS{T) - S) = 
r r 

r 

exp(cS) J2 Px,ACM)rR{r*)6{S{r*) +S)= eMS)PR{-S) . (26) 
r* 

In the first line of the derivation we used (I2ip . in the second we used (I25p and in the third 
we took into account the antisymmetric property of S(T) and the unicity of the assignment 
r ^ r*. This result is known under the generic name of fluctuation theorem, 

^^^^^ -exp(5) . (27) 



PRi-S) 



It is interesting to observe that this relation is not satisfied by the entropy production be- 
cause the inclusion of boundary term (|24|) in the total dissipation is required to respect the 
fluctuation symmetry. In what follows we discuss some of its consequences in some specific 
situations. 



Jarzynski equality. The nonequilibrium equality ()16p is just a consequence of ([Jj 
that is obtained by rewriting it as Pr{—S) = PpiS) exp(— 5) and integrating both sides 
of the equation from 5 = — ooto5 = oo. 

Linear response regime. Equation ((271) is trivially satisfied for 5 = if -Pf(O) = 
Pr{0). The process where Pp(^)(5) = 5{S) is called quasistatic or reversible. When S 
is different from zero but small {S < 1) we can expand ()27p around 5 = to obtain, 

SPF{S)=SeMS)PR{-S) 
{S)f = {{-S + S'))r + 0{S') 

{{S'))f(^r) = 2{S)f(^r) (28) 

where we used {S)f = {<S)r, valid up to second order in S. Note the presence of the 
subindex F{R) for the expectation values in the last line of (j28p which emphasizes the 
equality of these averages along the forward and reverse process. The relation (f28]l is a 
version of the fluctuation-dissipation theorem (FDT) valid in the linear response region 
and equivalent to the Onsager reciprocity relations [43j . 



3.3 Applications of the FT to nonequilibrium states 

The FT ()27p finds application in several nonequilibrium contexts. Here we describe specific 
results for transient and steady states. 



3.3.1 Nonequilibrium transient states (NETS) 

We will assume a system initially in thermal equilibrium that is transiently brought to a 
nonequilibrium state. We are going to show that, under such conditions, the entropy pro- 
duction (f22|) is equal to the heat delivered by the system to the sources. We rewrite ([22]) by 
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introducing the potential energy function (S'a(C), 



= "^P^ ^'^^^^ = exp(-GA(C) + g,) (29) 

where Zx = I^c 6^p(~^a(C)) = exp(— ^a) is the partition function and Qx is the thermody- 
namic potential. The existence of the potential Gx{C) and the thermodynamic potential Qx 
is guaranteed by Boltzmann-Gibbs ensemble theory. For simplicity we will consider here the 
canonical ensemble where the volume V, the number of particles N and the temperature T are 
fixed. Needless to say that the following results can be generalized to arbitrary ensembles. In 
the canonical case Gx{C) is equal to Ex{C)/T where Ex{C) is the total energy function (that 
includes the kinetic plus the potential energy terms). Gx is equal to Fx{V,T, N)/T where Fx 
stands for the Helmholtz free energy. 

With these definitions the entropy production (f22]l is given by, 



M-l , A/-1 



Sp{T) = J2 {Gx,{Ck) - Gx,{Ck+i)) = ^ E {ExACk) - Ex^Ck+i)) ■ (30) 



T 

k=0 k=0 



For the boundary term let us take 6(C) = Pxlj{C) 



B{T) = log(P,7(Co)) - logiP^iCM)) = 
= Gxm{C-m) - Gxq{Cq) - Qxm + Gxo = 

= ^ {Ex,, {Cm) - Ex, (Co) - Fx,, + Fx, ) (31) 
The total dissipation (|25|) is then equal to, 

S{T) = Sp{T) + ^{Ex,,{Cm) - Ex,{Co) - Fx,, + Fx,) (32) 

which can be rewritten as a balance equation for the variation of the energy Ex{C) along a 
given path, 

AE{T) = Ex„{Cm) - Ex,{Co) = TS{T) + AF - TSp{T) . (33) 

where AF = Fx,, — Fx, ■ This is the first law of thermodynamics where we have identified the 
term in the Ihs with the total variation of the internal energy A£'(r). Whereas TS{T) + AF 
and TSp{T) are identified with the mechanical work exerted on the system and the heat 
delivered to the bath respectively, 

A^(r) = W{T) - Q{T) (34) 
w{r) = TS{r) + AF (35) 
Q(T) = TSpiT) . (36) 

By using ()30p we obtain the following expressions for work and heat, 

WiT) = J2 {Ex,+ACk+i) - Ex,{Ck+i)) (37) 

k=0 
M-l 

Q(r) = E [Ex, (Cfc) - Ex, (Ck+i)) . (38) 

fe=0 
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The physical meaning of both entropies is now clear. Whereas Sp stands for the heat trans- 
ferred by the system to the sources (f36]l . the total dissipation term TS (f35]) is just the difference 
between the total mechanical work exerted upon the system, M^(r), and the reversible work, 
Wrev = AF. It is customary to define this quantity as the dissipated work, Wdiss; 

Wdiss{r) = TS{r) = w{r)-AF = w{r)-Wrev ■ (39) 



The nonequilibrium equality (jl6p becomes the nonequilibrium work relation originally derived 
by Jarzynski using Hamiltonian dynamics [31] . 

{e.^{-^)) = l or (exp(-f))=exp(-^) . (40) 

This relation is called the Jarzynski equality (hereafter referred as JE) and can be used to 
recover free energies from nonequilibrium simulations or experiments (see Sec. I4.2.2]) . The FT 
(j27p becomes the Crooks fluctuation theorem (hereafter referred as CFT) |441 145j . 

or ^^-expf^^^^l (41) 

The second law of thermodynamics W > AF also follows naturally as a particular case of 
(fTSl) by using (I39|40p . Note that for the heat Q a general relation equivalent to ([iT]) does not 
exist. We mention three aspects of the JE and the CFT. 

• The fluctuation-dissipation parameter R. In the limit of small dissipation Wdiss 
the linear response result ([^5]) holds. It is then possible to introduce a parameter ,R, 
that measures deviations from the linear response behavioi0. It is defined as, 

R = (42) 

where o"^ =< W'^ > — < >^ is the variance of the work distribution. In the limit 
W^diss — > 0, a second order cumulant expansion in (pUj) shows that R is equal to 1 and 
(I28p holds. Deviations from i? = 1 are interpreted as deviations of the work distribution 
from a Gaussian. When the work distribution is non-Gaussian the system is far from 
the linear response regime and ()28p is not satisfied anymore. 



• The Kirkwood formula. A particular case of the JE (I40p is the Kirkwood formula 
|46[ Wf\ . It corresponds to the case where the control parameter only takes two values 
Aq and Ai. The system is initially in equilibrium at the value Aq and, at an arbitrary 
later time t, the value of A instantaneously switches to Ai. In this case (|37p reads, 

W{T) = AEx{e) = ExAC)-Ex,{C) . (43) 

In this case a path corresponds to a single configuration, F = C, and (jlO]) becomes, 



exp(-^)^exp(-f) (44) 

the average (..) is taken over all configurations C sampled according to the equilibrium 
distribution taken at Aq, P'^{C). 



^Sometimes R is called fluctuation-dissipation ratio, not to be confused with the identically called but 
different quantity introduced in glassy systems, see Sec l6.2l that quantifies deviations from the fluctuation- 
dissipation theorem that is valid in equilibrium. 
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Heat exchange between two bodies. Suppose that we take two bodies initiahy at 
equihbrium at temperatures Th, Tc where Th,Tc stand for a hot and a cold temperature. 
At time t = we put them in contact and ask about the probabihty distribution of heat 
flow between them. In this case, no work is done between the two bodies and the heat 
transferred is equal to the energy variation of each of the bodies. Let Q be equal to the 
heat transferred from the hot to the cold body in one experiment. It can be shown |48j 
that in this case the total dissipation S is given by, 

5 = Q(l - (45) 

and the equality (fiOl) reads, 

(exp(-Q(l-^)) = l) (46) 

showing that, in average, net heat is always transferred from the hot to the cold body. 
Yet, sometimes, we also expect some heat to flow from the cold to the hot body. Again, 
the probability of such events will be exponentially small with the size of the system. 



3.3.2 Nonequilibrium steady states (NESS) 

Most investigations in nonequilibrium systems were initially carried out in NESS. It is widely 
believed that NESS are among the best candidate nonequilibrium systems to possibly extend 
Boltzmann-Gibbs ensemble theory beyond equilibrium \49\ I50j. 

We can distinguish two types of NESS: Time-dependent conservative (C) systems and non- 
conservative (NC) systems. In the C case the system is acted by a time-dependent force that 
derives from an external potential. In the NC case the system is driven by (time-dependent 
or not) non-conservative forces. In C systems the control parameter A has the usual meaning: 
it specifies the set of external parameters that, once fixed, determine an equilibrium state. 
Examples are: a magnetic dipole in an oscillating field (A is the value of the time-dependent 
magnetic field); a bead confined in an moving optical trap and dragged through water (A is 
the position of the center of the moving trap); a fluid sheared between two plates (A is the 
time-dependent relative position of the upper and lower plates). In C systems we assume local 
detailed balance so ([8]) still holds. 

In contrast to the C case, in NC systems the local detailed balance property, in the form 
of ([H]), does not hold because the system does not reach thermal equilibrium but a stationary 
or steady state. It is then customary to characterize the NESS by the parameter A and 
the stationary distribution by P^^{C). NESS systems in the linear regime (i.e. not driven 
arbitrarily far from equilibrium) satisfy the Onsager reciprocity relations where the fluxes are 
proportional to the forces. NESS can be maintained, either by keeping constant the forces 
or the fluxes. Examples of NC systems are: the flow of a current in an electric circuit (e.g. 
the control parameter A is either the constant current, I, flowing through the circuit or the 
constant voltage difference, AV); a Poiseuille fluid flow inside a cylinder (A could be either 
the constant fluid flux, or the pressure difference, AP); heat flowing between two sources 
kept at two different temperatures (A could be either the heat flux, Jq, or the temperature 
gradient, AT); the particle exclusion process (A = are the rates of inserting and removing 
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particles at both ends of the chain). In NESS of NC type the local detailed balance property 
([H]) still holds but replacing P^'^{C) by the corresponding stationary distribution, P|^(C), 

In a steady state in a NC system A is maintained constant. Because the local form of detailed 
balance (1470 holds the main results of Sec. [3] follow. In particular, the nonequilibrium equality 
()16p and the FT (j27p are still true. However there is an important difference. In steady states 
the reverse process is identical to the forward process PpiS) = Pr{S) because A is maintained 
constant. Therefore (fT6l) and the FT (1271) become, 



(exp(-5)) = 1 (48) 
^^^^ -exp(cS) . (49) 



P{-S) 



We can now extract a general FT for the entropy production Sp in NESS. Let us assume that, 
in average, Sp grows linearly with time, i.e. Sp ^ B for large t. Because S = Sp + B (|23ll . 
in the large t limit fluctuations in S are asymptotically dominated by fluctuations in Sp. In 
average, fluctuations in Sp grow like \/t whereas fluctuations in the boundary term are finite. 
By taking the logarithm of (|49p and using (j23p we obtain, 

S = log{PiS))-log{Pi-S))^Sp + B = logiPiSp + B))-logiP{-Sp-B)) . (50) 

In NESS the entropy produced, Sp{T), along paths of duration t is a fluctuating quantity. 
Expanding ([50l) around Sp we get, 

- ^°Hp(Z5^J + P(^ + 7^ - V (^1) 

The average entropy production (Sp) is defined by averaging Sp along an infinite number of 
paths. Dividing (fST]) by (Sp) we get, 

3l = ^ log(^i^] + + _ i] r52) 

{Sp) {Sp) ^\P{-Sp)J^ {Sp)\P{Sp)^ P{-Sp) ) ^ ^ 

We introduce a quantity a that is equal to the ratio between the entropy production and its 
average value, a = We can define the function. 

Equation (j52]) can be rewritten as. 



(S,) l P(Sp) + P(-S,) 

In the large time limit, assuming that log(P(S'p)) ~ t, and because B is finite, the second 
term vanishes relative to the first and /t(a) = + 0(1/^). Substituting this result into (|53p we 
find that an FT holds in the large t limit. However, this is not necessarily always true. Even 
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for very large t there can be strong deviations in the initial and final state that can make the 
boundary term B large enough to be comparable to {Sp). In other words, for certain initial 
and/or final conditions, the second term in the rhs of (I54p can be on the same order and 
comparable to the first term, a. The boundary term can be neglected only if we restrict the 
size of such large deviations, i.e. if we require \a\ < a* where a* is a maximum given value. 
With this proviso, the FT in NESS reads. 



In general it can be very difficult to determine the nature of the boundary terms. A specific 
result in an exactly solvable case is discussed below in Sec. 14.1.21 Eg. ([55]) is the Gallavotti- 
Cohen FT derived in the context of deterministic Anosov systems |28J- In that case Sp stands 
for the so called phase space compression factor. It has been experimentally tested by Ciliberto 
and coworkers in Rayleigh-Bernard convection and turbulent flows Similar relations 
have been also tested in athermal systems, e.g. in fluidized granular media [53] or the case of 
two- level systems in fluorescent diamond defects excited by light [M]. 

The FT (I27p also describes fluctuations in the total dissipation for transitions between 
steady states where A varies according to a given protocol. In that case, the system starts at 
time in a given steady state, P|^(C), and evolves away from that steady state at subsequent 
times. The boundary term for steady state transitions is then given by. 



where we have chosen the boundary function 6(C) = (C). In that case the total dissipation 
is antisymmetric under time-reversal and (I27p holds. Only in cases where the reverse process is 
equivalent to the forward process (j49p remains an exact result. Transitions between nonequi- 
librium steady states and expressions for the function S have been considered by Hatano and 
Sasa in the context of Langevin systems [55j. 

4 Examples and applications 

In this section we analyze in detail two cases where analytical calculations can be carried 
out and FTs have been experimentally tested. We have chosen two examples: one extracted 
from physics, the other from biology. We first analyze the bead in a trap to later consider 
single molecule pulling experiments. These examples show that there are lots of interesting 
observations that can be made by comparing theory and nonequilibrium experiments in simple 
systems. 

4.1 A physical system: a bead in an optical trap 

It is very instructive to work out in detail the fluctuations of a bead trapped in a moving 
potential. This case is of great interest for at least two reasons. First, it provides a simple 
example of both a NETS and a NESS that can be analytically solved in detail. Second, it 
can be experimentally realized by trapping micron-sized beads using optical tweezers. The 
first experiments studying nonequilibrium fluctuations in a bead in a trap were carried out by 
Evans and collaborators |56j and later on extended in a series of works [57\ I58j. Mazonka and 




I ^ * 
a\ < a 



(55) 



BiT) = log(P|f,(Co)) - log(P,- (Cm)) 



(56) 
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Jarzynski [59] and later Van Zon and Cohen [BOj [6T1 [62] have carried out detailed theoretical 
calculations of heat and work fluctuations. Recent experiments have also analyzed the case 
of a particle in a non-harmonic optical potential p3]. These results have greatly contributed 
to clarify the general validity of the FT and the role of the boundary terms appearing in the 
total dissipation S. 

The case of a bead in a trap is also equivalent to the power fluctuations in a resistance in 
an RC electrical circuit [64] (see Figure H}. The experimental setup is shown in Figure O A 
micron-sized bead is immersed in water and trapped in an optical well. In the simplest case 
the trapping potential is harmonic. Here we will assume that the potential well can have an 
arbitrary shape and carry out specific analytical computations for the harmonic case. 

Let X be the position of the bead in the laboratory frame and U{x — x*) the trapping 
potential of a laser focus that is centered at a reference position x*. For harmonic potentials 
we will take U{x) = {1/2)kx'^. By changing the value of x* the trap is shifted along the x 
coordinate. A nonequilibrium state can be generated by changing the value of x* according 
to a protocol x*{t). In the notation of the previous sections, A = x* is the control parameter 
and C = X is the configuration. A path F starts at x(0) at time and ends at x{t) at time t, 
F = {x(s);0 < s < t}. 

At low Reynolds number the motion of the bead can be described by a one-dimensional 
Langevin equation that contains only the overdamping term, 

7i = (x) + 7? ; ivitHs)) = 2T^5{t - s) (57) 

where x is the position of the bead in the laboratory frame, 7 is the friction coefficient, fx*{x) 
is a conservative force deriving from the trap potential U{x — x*), 

fAx) = -{U{x - X*))' = (58) 

and r/ is a stochastic white noise. 

In equilibrium x* (t) = x* is constant in time. In this case, the stationary solution of the 
master equation is the equilibrium solution 

/ U(x-x*)\ / U(x-x*)\ 

peq/ X ^ exp(- \ > ) ^ exp(- \ ' ) 

Jdxexp{-U{x-x*)/T) Z ^ ' 

where Z = ^ dx exp{—U{x)/T) is the partition function that is independent of the reference 
position X*. Because the free energy F = —Tlog{Z) does not depend on the control parameter 
X*, the free energy change is always zero for arbitrary translations of the trap. 

Let us now consider a NESS where the trap is moved at constant velocity, x*{t) = vt. 
It is not possible to solve the Fokker-Planck equation to find the probability distribution in 
the steady state for arbitrary potentials. Only for harmonic potentials, U{x) = kx^/2, the 
Fokker-Planck equation can be solved exactly. The result is, 

, /2^TT^-^ / K(x-x*(t) + ^f^ , , 

P-(x) = (^) %xp(-^ ^ ^) (60) 

Note that the steady-state solution ([HU]) depends explicitly on time through x*(t). To obtain 
a time-independent solution we must change variables x — > x — x*(t) and describe the motion 
of the bead in the reference frame that is solidary and moves with the trap. We will come 
back to this problem later in Sec. 14.1.31 
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4.1.1 Microscopic reversibility 

In this section we show that the Langevin dynamics ()57p satisfies the microscopic reversibihty 
assumption or local detailed balance ([H]). We recall that x is the position of the bead in the 
laboratory frame. The transition rates Wx* {x — > x') for the configuration x at time t to change 
to x' at a later time t + At can be computed from ()57p . We discretize the Langevin equation 
[65] by writing, 

,.^,^f(^M+^r + 0{(Atf) (61) 

where r is a random Gaussian number of zero mean and unit variance. For a given value of 
X, the distribution of values x' is also a Gaussian with average and variance given by, 

^ = /(^^^!M + O ((At)2) (62) 

_ 2' 

7 

and therefore. 



al, = {x'Y - {{x')f = ^ + 0({Atf) (63) 



(x' - X+ /(^-^'*)^* )2 

Wx*{x ^ x') = (27r4)-^ exp( ^-^-^^ ) . (64) 

From ()64p we compute the ratio between the transition probabilities to first order in At, 

Wx*{x^x') / {x' -x){f{x-x*)+ f{x' -x*))\ 

-exp — . (65) 



Wx*{x'^x) '\ 2T 
We can now use the Taylor expansions, 

U{x' - x*) = U{x - X*) - f{x - x*){x' -x) + 0{{x' - xf) (66) 
U{x - X*) = U{x' - X*) - f{x' - x*){x - x') + 0{{x' - xf) (67) 

and subtract both equations to finally obtain, 

{x' - x){f{x - X*) + f{x' - X*)) = 2{U{x' - X*) - U{x - X*)) (68) 

which yields, 

Wx*{x^x') _ ( U{x' -x*)-U{x-x*) ^ _ P:?(xO 

h;,*(x' ^ x) " """PV T ) ~ p:?(x) 

which is the local detailed balance assumption 

4.1.2 Entropy production, work and total dissipation 

Let us consider an arbitrary nonequilibrium protocol x*{t) where v{t) = x*(t) is the velocity 
of the moving trap. The entropy production for a given path, F = {x(s);0 < s < t} can be 
computed using ([^^ . 

S,(r) = ^,M.(,)( ^)_,,, . (70) 
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We now define the variable y{t) = x{t) — x*{t). From (j59p we get I, 

5p(r) = i ds±{s)f{x{s) - x*{s)) = ^ ds{y{s) + v{s))f{y{s)) = (71) 
l(r^./(.)+^I(.)/(.))^^^™^ (72) 

with ^ 

AC/ = ;7(x(t)-x*(t))-C/(2;(0)-x*(0)) ; W{r) = [ dsv{s)f{s) . (73) 

JO 

where we used (I58p in the last equality of ()72p . AC/ is the variation of internal energy between 
the initial and final positions of the bead and Vl^(r) is the mechanical work done on the bead 
by the moving trap. Using the first law AU = W — Q we get, 

Sp{r) = ^ (74) 

and the entropy production is just the heat transferred from the bead to the bath divided by 
the temperature of the bath. 

The total dissipation S ([23|) can be evaluated by adding the boundary term (p^ to the 
entropy production. For the boundary term we have some freedom as to which function b we 
use in the rhs of (j24p . 

S(r)=log(p,*(0)(:E(0))) -log(6(x(t))) . (75) 

Because we want S to be antisymmetric against time-reversal there are two possible choices 
for the function b depending on the initial state, 

• Nonequilibrium transient state (NETS). Initially the bead is in equilibrium and 
the trap is at rest in a given position x*(0). Suddenly the trap is set in motion. In this 
case we choose b{x) = P^^^^^(x) and the boundary term (p^ reads, 

B{r) = log(p:?(o)(x(0))) - log(p,^?(,)(x(t))) . (76) 
By inserting (j59p we obtain, 

B{r) = ^mxit) - x*{t)) - U{x{0) - x*(0))) = ^ (77) 

and S = Sp + B = {Q + AU)/T = W/T so the work satisfies the nonequilibrium equality 
dUD and the FT 1^, 

Note that in the reverse process the bead starts in equilibrium at the final position x*{t) 
and the motion of the trap is reversed {x*)^{s) = x*{t — s). The result ()78p is valid for 



^Note that i, the velocity of the bead, is not well defined in (|70I72|I . However, dsx{s) — dx it is. Yet we 
prefer to use the notation in terms of velocities just to make clear the identification between the time integrals 



in (|70l72p and the discrete time-step sum in (f22l 
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arbitrary potentials U{x). In general, the reverse work distribution PuiW) will differ 
from the forward distribution PpiW). Only for symmetric potentials U{x) = U{—x) 
both work distributions are identical [66]. Under this additional assumption (I78p reads, 



P(-W) T 
Note that this is a particular case of the CFT (|4ip with AF = 0. 

Nonequilibrium steady state (NESS). If the initial state is a steady state, Paq (Co) = 
P^^j.Q^(x), then we choose h{x) = P^l^^-^{x). The boundary term reads, 

B{r) = log(p-(o)(x(0))) - log(p-(,)(x(t))) (80) 



Only for harmonic potentials we exactly know the steady state solution ()60p so we can 
write down an explicit expression for B, 

where AU is defined in (173]) and A/ = /^.*(t)(x(t)) — /x*(o) (^(0))- The total dissipation 
is given by, 

Q + AU v-/Af _ W v-/Af 

It is important to stress that ([82|) does not satisfy (j48l49p because the last boundary 
term in the rhs of (|82p [v^Af/nT) is not antisymmetric against time reversal. Van Zon 
and Cohen [60^ [6H [62] have analyzed in much detail work and heat fluctuations in the 
NESS. They find that work fluctuations satisfy the exact relation, 

P{W) (W 1 ^ 

~exp — — — — - (83) 



5 = 5^ + S = ^lX^_IiI^ = ^_IiI^ . (82) 



-Vr l + i(exp(-i)-i: 

where t is the time window over which work is measured and r is the relaxation time of 
the bead in the trap, r = ^/k. Note that the FT (|79p is satisfied in the limit r/t — > 0. 
Corrections to the FT are on the order of r/t as expected (see the discussion in the last 
part of Sec. I3.3.2p . Computations can be also carried out for heat fluctuations. The 
results are expressed in terms of the relative fluctuations of the heat, a = The 
large deviation function /t(a) (|53p has been shown to be given by. 



lim ft{a) = a (0 < a < 1); 

>oo 

lim ft{a) = a - (a - if/i (1 < a < 3); 

hm ft{a) = 2 (3 < a); (84) 

t — >oo 

and ft{—a) = —ft{o)- Very accurate experiments to test (I83|84p have been carried out 
by Garnier and Ciliberto who measured the Nyquist noise in an electric resistance |67] . 
Their results are in very good agreement with the theoretical predictions which include 
corrections in the convergence of (|84p on the order of 1/t as expected. A few results are 
shown in Figure Hi 
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Figure 4: Heat and work fluctuations in an electrical circuit (left). PDF distributions (center) 
and verification of the FTs l^83\84\) (right). Figure taken from \61^. 



4.1.3 Transitions between steady states 

Hatano and Sasa [55j have derived an interesting result for nonequilibrium transitions between 
steady states. Despite of the generahty of Hatano and Sasa's approach, exphcit computations 
can be worked out only for harmonic traps. In the present example the system starts in a 
steady state described by the stationary distribution ([BU]) and is driven away from that steady 
state by varying the speed of the trap, v. The stationary distribution can be written in the 
frame system that moves solidary with the trap. If we define y{t) = x — x*{t) then (|6U|) 
becomes, 

Pf{y) = (2vrr/K)-| exp(- ^^^+^^^' ) . (85) 

Note that, when expressed in terms of the reference moving frame, the distribution in the 
steady state becomes stationary or time independent. The transition rates (fMll can also be 
expressed in the reference system of the trap. 



^ y' 



AirTAt^ 



7 



) 2 exp(- 



7(y' 



y + {v + ^y)Atf 



4TAt 







(86) 



where we have used f{x — x*) = f{y) = —Ky. The transition rates }Vv{y y') now depend on 
the velocity of the trap. This shows that, for transitions between steady states, \ = v plays 
the role of the control parameter, rather than the value of x*. A path is then defined by the 
evolution T = {y(s);0 < s < t} whereas the perturbation protocol is specified by the time 
evolution of the speed of the trap {v{s);d < s <t}. 

The rates Wv{y y') satisfy the local detailed balance property ([^7j) . From (j86]) and (j85]) 
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we get (in the limit At 0), 

W,{y'^y) P-(y) 



(87) 

exp( -;^(y'^ - y^) - f (y' - y)) = exp(-(^ - ^)) (88) 



Note that the exponent in the rhs of (|88|) is equal to the boundary term (|81|) . In the reference 
system of the trap we can then compute the entropy production Sp and the total dissipation 
S. From either or ([55]) and using ([55]) we get, 

'log(P-(y)x AC/ , 7 '■i 



= -^(aU-^{A{vF)) + ^ fdsi{s)F{s)) (90) 

1 ^ K K Jo / 



where we integrated by parts in the last step of the derivation. For the boundary term ([56 
we get, 



B(r) = log(P„7„,(9(0))) - log(F.!^„(!,(())j = (91) 



TV ' " 2k 

where we used (I85p . By adding ([90p and ([92p we obtain the total dissipation, 

K io 



S = Sp + B = ^(A{^)-l f dsv{s)F{s)) = (93) 
1 ^ Ik k Jo ' 



.1 

K Jo 



t 

dsij{s){F{s) --/v{s)) (94) 



The quantity S (called Y by Hatano and Sasa) satisfies the nonequilibrium equality ([16p 
and the FT ([27p . Only for time reversal invariant protocols, v^{s) = v{t — s), we have 
Pf{<S) = Pr{S), and the FT ([lU]) is also valid. We emphasize two aspects of ([M]) . 

• Generalized second law for steady state transitions. From the inequality (|18p 
and ([9l]) we obtain, 

7 



r dsHs)F{s) < A{:^) (95) 

which is reminiscent of the Clausius inequality Q > —TAS where the average dissipation 
rate Pdiss = 7^^ plays the role of a state function similar to the equilibrium entropy. 
In contrast to the Clausius inequality the transition now occurs between steady states 
rather than equilibrium states |68j . 

Non-invariance of entropy production under Galilean transformations. In 

steady states where v = 0, Sp ([90]) becomes a boundary term and 5 = ([M]) . However 
we saw in ([74p that Sp is equal to the heat delivered to the environment (and therefore 
proportional to the time elapsed t) whereas now Sp is a boundary term that does not 
grow with t. This important difference arises from the fact that the entropy production 
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Figure 5: (Left) Bead confined in a moving optical trap. (Right) Total entropy S distributions 
(h,c,d) for the velocity protocols shown in (a). Figure taken from J6^ . 



is not invariant under Galilean transformations. In the reference of the moving trap 
the bath is moving at a given speed which impedes to define the heat in a proper way. 
To evaluate the entropy production for transitions between steady states one has to 
resort to the description where x* is the control parameter and x*{t) = f^dsvi^s) is the 
perturbation protocol. In such description the results ()73p and ()74p are still valid. 

These results have been experimentally tested for trapped beads accelerated with different 
velocity protocols [68]. Some results are shown in Figure O 



4.2 A biological system: pulling biomolecules 

The development of accurate instruments capable of measuring forces in the piconewton range 
and extensions on the order of the nanometer gives access to a wide range of phenomena in 
molecular biology and biochemistry where nonequilibrium processes that involve small energies 
on the order of a few ksT are measurable (see Sec. [2|). From this perspective the study of 
biomolecules is an excellent playground to explore nonequilibrium fluctuations. The most 
successful investigations in this area have been achieved in single molecule experiments using 
optical tweezers |69j . In these experiments biomolecules can be manipulated one at a time 
by applying mechanical force at their ends. This allows us to measure small energies under 
varied conditions opening new perspectives in the understanding of fundamental problems in 
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Figure 6: Pulling single molecules. (A) Stretching DNA; (B) Unzipping DNA; (C) Mechanical 
unfolding of RNA and (D) Mechanical unfolding of proteins. 

biophysics, e.g. the folding of biomolecules [TOl CU [72] . The field of single molecule research 
is steadily growing with new molecular systems being explored that show nonequilibrium 
behavior characteristic of small systems. The reader interested in a broader view of the area 
of single molecules research is suggested to have a look at reference [73] . 

4.2.1 Single molecule force experiments 

In single molecule force experiments it is possible to apply force on individual molecules by 
grabbing their ends and pulling them apart [74L [75l 1761 [77] . Examples of different ways in 
which mechanical force is applied to single molecules are shown in Figure [6j In what follows 
we will consider single molecule force experiments using optical tweezers, although everything 
we will say extends to other force techniques (AFM, magnetic tweezers or biomembrane force 
probe, see [7H]) In these experiments, the ends of the molecule (for example DNA |^]) are 
labeled with chemical groups (e.g. biotin or digoxigenin) that can bind specifically to their 
complementary molecular partners (e.g. streptavidin or anti-digoxigenin respectively). Beads 
are then coated with the complementary molecules and mixed with the DNA in such a way 
that a tether connection can be made between the two beads through specific attachments. 
One bead is in the optical trap and used as a force sensor. The other bead is immobilized 
on the tip of a micropipette that can be moved by using a piezo-controlled stage to which 
the pipette is attached. The experiment consists in measuring force-extension curves (FECs) 
by moving the micropipette relative to the trap position [TIJ'. In this way it is possible to 
investigate the mechanical and elastic properties of the DNA molecule [SUj 181). 

Many experiments have been carried out by using this setup: the stretching of single DNA 
molecules, the unfolding of RNA molecules or proteins and the translocation of molecular 
motors (Figure [2]). Here we focus our attention in force experiments where mechanical work 
can be exerted on the molecule and nonequilibrium fluctuations measured. The most successful 
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studies along this line of research are the stretching of small domain molecules such as RNA 
[82j or protein motifs [83]. Small RNA domains consist of a few tens of nucleotides folded 
into a secondary structure that is further stabilized by tertiary interactions. Because an RNA 
molecule is too small to be manipulated with micron-sized beads, it has to be inserted between 
molecular handles. These act as polymer spacers that avoid non-specific interactions between 
the bead and the molecule as well as the contact between the two beads. 

The basic experimental setup is shown in Figure [71 We also show a typical FEC for an 
RNA hairpin and a protein. Initially the FEC shows an elastic response due to the stretching 
of the molecular handles. Then, at a given value of the force, the molecule under study unfolds 
and a rip is observed in both force and extension. The rip corresponds to the unfolding of the 
small RNA/protein molecule. The molecule is then repeatedly stretched and relaxed starting 
from the equilibrated native/extended state in the pulling/relaxing process. In the pulling 
experiment the molecule is driven out of equilibrium to a NETS by the action of a time 
dependent force. The unfolding/refolding reaction is stochastic, the dissociation/formation of 
the molecular bonds that maintain the native structure of the molecule is overdamped by the 
Brownian motion of the surrounding water molecules [Slj . Every time the molecule is pulled 
different unfolding and refolding values of the force are observed (inset of Figure [Tp). The 
average value of the force at which the molecule unfolds during the pulling process increases 
with the loading rate (roughly proportional to the pulling speed) in a logarithmic way as 
expected for a two-state process (see the discussion at the end of Sec. 15.2.11 and equation 
(fM ). 

Single molecule pulling experiments can be described with the formalism developed in 
Sec. I3.3.1[ In the simplest setting the configurational variable C corresponds to the molecular 
extension of the complex (handles plus inserted molecule) and the control parameter A is 
either the force / measured in the bead or the molecular extension of the system, x. For small 
enough systems the thermodynamic equation of state is dependent on what is the variable 
that is externally controlled [86]. In the actual experiments, the assumption that either the 
force or the extension are controlled is just an approximation. Neither the molecular extension 
or the force can be really controlled in optical tweezers [57]. For example, in order to control 
the force a feedback mechanism must operate at all times. This feedback mechanism has a 
time delay response so the force is never really constant [88^189]. By assuming a constant force 
we are neglecting some corrections in the analysi^. Under some conditions these corrections 
are shown to be unimportant (see below). Let us now consider that the force acting on the 
inserted molecule is controlled (the so called isotensional ensemble). For a molecule that is 
repeatedly pulled from a minimum force value, /mim up to a maximum force, /max, the work 
([371) along a given path is given by, 

/■/max dE(x f) /-/max 

WfiT) = / df^^fl^ = - / x{f)df (96) 

/min /min 

where the energy function is given by E{x, f) = E{x, 0) — fx, E{x, 0) being the energy function 
of the molecule at zero force. The subindex / in Wf is written to underline the fact that we 
are considering the isotensional case where / is the control parameter. The Jarzynski equality 

''By using two traps, it is possible to maintain a constant force (9^. This is also possible with magnetic 
tweezers. However, because of the low stiffness of the magnetic trap, the spatial resolution along individual 
trajectories is limited by thermal fluctuations to 10 nanometers or more. 
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Figure 7: Mechanical unfolding of RNA molecules (A,B) and proteins (C,D) using optical 
tweezers. (A) Experimental setup in RNA pulling experiments. (B) Pulling cycles in the 
caconical hairpin CD4 and force rip distributions during the unfolding and refolding at three 
pulling speeds. (C) Equivalent setup in proteins. (D) Force extension curve when pulling the 
protein RNAseH. Figure (B) taken from ]83^ . Figures (A,D) are a courtesy from C. Cecconi 
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(j40p and the FT (j4ip hold with AF equal to the free energy difference between the initial 
and final equilibrium states. We assume that the molecule is immersed in water at constant 
temperature T, pressure p and acted by a force /. The thermodynamic free energy F{T,p, f) 
in this description is the Legendre transform of the Gibbs free energy at zero force, ambient 
temperature T and pressure p, G{T,p) [9T| l92]. 

FiT,p,f) = G{T,p) - fx{T,p,f) ^ x{T,P,f) = . (97) 

We are interested in knowing the Gibbs free energy difference at zero force, AG, rather than 
the free energy difference AF between the folded state at /min and the unfolded-extended 
state at /max- We can express (|40|4ip in terms of G (rather than F) and define the corrected 
work Wj(r) along a path, 

Wf{r) = WfiT) + A(X/) = Wf{r) + (Xmax/max " 2;mm/min) (98) 

where the extensions Xmim 2;max are now fluctuating quantities evaluated at the initial and 
final times along each pulling. The corrected work VFj(r) includes an additional boundary 
term and therefore does not satisfy neither the JE and the CFT. If we now consider that 
X is the control parameter then we can define the equivalent of ()96p (the so-called isometric 
ensemble) , 

W^{T) = Wf{T) + A(x/) = f{x)dx (99) 



where now x is controlled and Xmin, a;max are fixed by the pulling protocol. Equations ()98|99p 
look identical, however they refer to different experimental protocols. Note that the term 
WfiT) appearing in (I99p is now evaluated between the initial and final forces at fixed initial 
and final times. Both works Wx,Wf satisfy the relations (|4U|4ip . For a reversible process 
where / is controlled we have Wj'^^ = AF whereas if x is controlled we have W^^^ = AG. 
In experiments it is customary to use (j99p for the work. First, because that quantity is more 
easily recognized as the mechanical work. Second, because it gives the free energy difference 
between the folded and the unfolded states at zero force, a quantity that can be compared 
with the value obtained from thermal denaturation experiments. 

In general neither the force or the molecular extension can be controlled in the experiments 
so both definitions ()96|98|99p result into approximations to the true mechanical work that 
satisfies (I40|4ip . The control parameter in single molecule experiments using optical tweezers 
is the distance between the center of trap and the immobilized bead [57]. Both the position 
of the bead in the trap and extension of the handles are fluctuating quantities. It has been 
observed [H311M1ES] that in pulling experiments the proper work that satisfies the FT includes 
some corrections to the expressions (I97I99P mainly due to the effect of the trapped bead. There 
are two considerations to take into account when analyzing experimental data: 

• Wx or Wf? Let us suppose that / is the control parameter. In this case the JE and CFT 
(|40|4ip are valid for the work ()96p . How large is the error that we make when we apply 
the JE using Wx instead? This question has been experimentally addressed by Ciliberto 
and coworkers [96 1 [971 who measured the work in an oscillator system with high precision 
(within tenths of A;_bT). As shown in (|99p the difference between both works is mainly 
a boundary term, A(x/). Fluctuations of this term can be a problem if they are on the 
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same order than fluctuations of Wx itself. For an harmonic oscillator of stiffness constant 
equal to k, the variance of fluctuations in fx are equal to k5{x'^), i.e. approximately on 
the order of ksT due to the fluctuation-dissipation relation. Therefore, for experimental 
measurements that do not reach such precision, Wx or Wf are equally good. 

The effect of the bead or cantilever. Hummer and Szabo [93j have analyzed the 
effect of a force sensor attached to the system (i.e. the bead in the optical trap or the 
cantilever in the AFM) in the work measurements. To this end they consider a simplified 
model of the experimental setup (Figure [8|). In such model the molecular system (that 
includes the molecule of interest -RNA or protein- and the handles) is connected to a 
spring (that models the trapped bead or the AFM cantilever) and the whole system is 
embedded in a thermal bath. The total extension of the molecular system is x but the 
control parameter is z = x + Xb where Xb is the position of the bead respect to the center 
of the trap. The total free energy of the system is given by F{x) + {1/2)kxI where F{x) 
is the free energy of the molecular system alone and k is the stiffness of the trap. The 
molecular extension x and the distance Xb are related by the force balance equation, 

f = nxb = (100) 

where we assume that the bead is locally equilibrated at all values of the nonequilibrium 
molecular extension x (this is a good approximation if, as often occurs, the bead relaxes 
fast enough compared to the typical time for the unfolding/refolding of the molecule). 
The mechanical work ()37p is then given by, 

f 



fdz = Wxir) + A[^) (101) 



where we used dz = dxb + dx and (llOOp . The difference between the proper work W and 
Wx is again a boundary term. Because z is the control parameter the JE and the CFT 
are valid for the work W but not for Wx- Again, the FT will not hold if fluctuations in 
the boundary term are important. The variance of these fluctuations is given by, 

where Kx is the stiffness of the molecular system [981 EZ] • Usually, ^> k so fluctuations 
in the boundary term are again smaller than ksT. In general, as a rule of thumb, we can 
say that it does not matter much which mechanical work we measure if we do not seek 
for free energy estimates with an accuracy less than UbT. This is true unless the bead 
(cantilever) does not equilibrate within the timescale of the experiments. This may be 
the case when k is too low and (jlOOp is not applicable. 



4.2.2 Free energy recovery 

As we already emphasized the JE (I40p and the FT (I4ip can be used to predict free energy 
differences. In single molecule experiments it is usually difficult to pull molecules in a reversible 
way due to drift effects in the instrument. It is therefore convenient to devise nonequilibrium 
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Figure 8: A molecular system of extension x is connected at its leftmost end to a bead trapped in 
an optical well (or to the tip of an AFM cantilever) and at its rightmost end to an immobilized 
surface (or a bead fixed to the tip of micropipette) . The position of the bead relative to the 
center of the trap, Xh, gives a read out of the acting force f = kx^. The control parameter in 
this setup is z = Xf, + X whereas both x^ and x are fluctuating quantities. 

methods (such as the JE or the CFT) to extract equihbrium free energies differences from data 
obtained in irreversible processes. The first experimental test of the JE was carried out by 
pulling RNA hairpins that are obtained as derivatives of the L21 Tetrahymena ribozyme |99j . 
In these experiments RNA molecules were pulled not too fast, the average dissipated work in 
such experiments was less Gfe^T and the work distributions turned out to be approximately 
Gaussian. Recent experiments have studied RNA molecules that are driven farther away from 
equilibrium in the non- linear regime. In such non-linear regime the average dissipated work 
is non-linear with the pulling speed [lOOj and the work distribution strongly deviates from a 
Gaussian |101j . In addition, these experiments have provided the first experimental test of 
the CFT (I4ip . These measurements have also shown the possibility to recover free energy 
differences by using the CFT with larger accuracy than that obtained by using the JE. There 
are two main predictions of the CFT (I4ip that have been scrutinized in these experiments, 

• Forward and reverse work distributions cross at W = AG. In order to obtain 
AG we can measure the forward and reverse work distributions Pf{W),Pr{—W) and 
look at the work value W* where they cross, Pf{W*) = Pji{—W*). According to (HTI) 
both distributions should cross at W* = AG independently of how far the system is 
driven out of equilibrium (i.e. independently of the pulling speed). Figure [9] shows 
experiments on a short canonical RNA hairpin CD4 (i.e. just containing Watson-Crick 
complementary base pairs) at three different pulling speeds which agree very well with 
the FT prediction. 

• Verification of the CFT. The CFT ([H]) can be tested by plotting log(PF(Ty) /Pr{-W)) 
as a function of W. The resulting points should fall in a straight line of slope 1 (in ksT 
units) that intersects the work axis at W = AG. Of course, this relation can be tested 
only in the region of work values along the work axis where both distributions (forward 
and reverse) overlap. An overlap between the forward and reverse distributions is hardly 
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Figure 9: (A) Structure of the canonical CD4 hairpin. (B) FECs at a loading rate of 1.7pN/s. 
(C) Unfolding and refolding work distributions at three loading rates (see inset). The unfolding 
and refolding work distributions cross at a value AG independent of the pulling speed as 
predicted by the CFT. Data correspond to 100,400 and 700 pulls for the lowest, medium and 
highest pulling speeds respectively. (D) Test of the CFT at the intermediate loading rate 
7.5pN/s for 4 different tethers. The trend of the data is reproducible from tether to tether and 
consistent with the CFT prediction. Figures taken from jlOlf . 
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Figure 10: (A) Secondary structure of the three way junction S15. (B) A few FECs for 
the wild type. (C) Unfolding /refolding work distributions for the wild type and the mutant. 
(Inset) Experimental verification of the validity of the CFT for the mutant where unfolding and 
refolding distributions overlap each other over a limited range of work values. Data correspond 
to 900 pulls for the wild type and 1200 pulls for the mutant. Figure taken from '1 01']. 

observed if the molecules are pulled too fast or if the number of pulls is too small. In 
such cases, other statistical methods (Bennet's acceptance ratio or maximum likelihood 
methods, Sec. I4.2.3P can be applied to get reliable estimates of AG. The validity of the 
CFT has been tested in the case of the RNA hairpin CD4 previously mentioned and the 
three way junction RNA molecule as well. Figures [9lC,D and HOC show results for these 
two molecules. 

In general, both the JE and the CFT are only valid in the limit of an infinite number 
of pulls. For a finite number of pulls, N , the estimated value for AG that is obtained by 
applying the JE is biased [102]. The free-energy estimate F^ for a given set, k, of N work 
values Wi, W2, is defined as. 




(103) 
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The free-energy bias is defined by averaging the estimator F^^ over an infinite number of sets, 

B(N) = hm \— Y fF] - AF (104) 

k=l 

where AF is the true free energy difference. The bias B{N) converges to for N ^ oo. 
However, it is of practical importance to devise methods to estimate how many pulls are 
required to obtain the Jarzynski free energy estimate F'^^ within a reasonable error far from 
the true value [10311100] . The bias is a complicated mathematical object because the Jarzynski 
average catches important contributions from large deviations of the work. As we will see later 
in Sec. 15.3.21 the bias is a large deviation function that requires specific mathematical methods 
to analyze its finite N behavior and large convergence. There we proof that, for large A'^, 
the bias decreases as 1/A^, a result that is known as Woods formula [103j . In the intermediate 
A^ regime the behavior of the bias is more complicated |104j . Free-energy recovery techniques 
are also used in numerical simulations to evaluate free-energy differences [1051 11061 11071 1108j 
and reconstruct free-energy profiles or potentials of mean-field force [1091 1110] . 



4.2.3 Efficient strategies and numerical methods 

An important question is to understand the optimum nonequilibrium protocol to recover free- 
energies using the JE given specific constraints in experiments and simulations. There are 
several considerations to take into account, 

• Faster or slower pulls?. In single molecule experiments tethers break often so it is not 
possible to repeatedly pull the same tether an arbitrary number of times. Analogously, 
in numerical simulations only a finite amount of computer time is available and only 
a limited number of paths can be simulated. Given these limitations, and for a given 
amount of available time, is it better to perform many fast pulls or a few number of 
slower pulls to recover the free energy difference using the JE?. In experiments, drift 
effects in the instrument always impose severe limitations to the minimum speed at 
which molecules can be pulled. To obtain good quality data it is advisable to carry out 
pulls as fast as possible. In numerical simulations the question about the best strategy 
for free energy recovery has been considered in several papers |1111 11121 11131 HO^ - The 
general conclusion that emerges from these studies is that, in systems that are driven 
far away from equilibrium, it is preferable to carry out many pulls at high speed than 
a few number of them at slower speeds. The reason can be intuitively understood. 
Convergence in the JE is dominated by the so-called outliers, i.e. work values that 
deviate a lot from the average work and are smaller than AF. The outliers contribute 
a lot to the exponential average (|40p . For higher pulling speeds we can perform more 
pulls so there are more chances to catch a large deviation event, i.e. to catch an outlier. 
At the same time, because at higher speeds the pulling is more irreversible, the average 
dissipated work becomes larger making the free energy estimate less reliable. However 
the contribution of the outliers that is required to recover the right free-energy largely 
compensates the slow convergence due to the increase of the average dissipated work. 
We should also mention that periodically oscillating pulls have been also considered, 
however it is unclear whether they lead to improved free energy estimates |1141 1115] . 
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• Forward or reverse process?. Suppose we want to evaluate the free energy difference 
between two states, A and B, by using the JE. Is it better to estimate AF by carrying 
out irreversible experiments from A to B, or is it better to do them from B to A? In- 
tuitively it seems natural that the less irreversible process among the two (forward and 
reverse), i.e. the one with smaller dissipated work VFdissj is also the most convenient in 
order to extract the free energy difference. However this is not the case. In general, a 
larger average dissipate work implies a larger work variance (I42p . i.e. larger fluctuations. 
The larger are the fluctuations, the larger the probability to catch a large deviation that 
contributes to the exponential average. It seems reasonable that, if outliers contribute 
much more to obtain the correct free energy difference than proper tuning of the average 
value of the work does, then the process that more fluctuates, i.e. the more dissipative 
one, is the preferred process in order to efficiently recover AF. This result was antic- 
ipated in |116j and analyzed in more detail in |117j . For Gaussian work distributions, 
the minimum number of pulls, N*, required to efficiently recover free energy differences 
within IksT by using the JE grows exponentially with the dissipated work along the 
nonequilibrium process [100] . However, for general work distributions, the value of Np^j^-^ 
along the forward (reversed) process depends on the average dissipated work along the 
reverse (forward) process |117j . This implies that. 



and the process that dissipates most between the forward and the reverse is the most 
convenient to efficiently recover AF. 

Until now we discussed strategies to recover free energy differences using the JE. We might 
be interested in free energy recovery by combining the forward and reverse distributions at the 
same time that we use the CFT. This is important in either experiments [lOlj or simulations 
[118\ I119j where it is convenient and natural to use data from the forward and reverse process. 
The best strategy to efficiently recover free energies using the forward and reverse processes 
was proposed by C. Bennett long time ago in the context of equilibrium sampling |120j . The 
method has been later extended by Crooks to the nonequilibrium case [45J and is known as 
Bennett's acceptance ratio method. The basis of the method is as follows. Let us multiply 
both sides of ([^T]) by the function g^{W), 



where gfj^{W) is an arbitrary real function that depends on the parameter /x. Integrating both 
sides between W = —oo and W = oo gives. 




(105) 




(106) 




(107) 



where {..■)[f,r) denote averages over the forward and reverse process. Taking the logarithm 
at both sides we have. 
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where we have defined, 



zr{^i) = \og({g^{W))R) (109) 



zf{ix) = log((5^(W^)exp(-^))^) . (110) 

Equation (llOSp imphes that the difference between both functions zf,zr must be a constant 
over all fj, values. The question we would like to answer is the following. Given a finite 
number of forward and reverse pulls, what is the optimum choice for Qf^iW) that gives the 
best estimate (jlOSp for AF? For a finite number of experiments Np,Nfi along the forward 
and reverse process we can write for any observable A, 

^ ^F{R) 



{A{W))F^n) = ^fzz: E MW^) (111) 
Equation ()107p yields an estimate for AF, 



i=l 



{AFr' =T{log{{g,{W))n) -log{{g,{W)e^p(-^))F)) . (112) 



Minimization of the variance. 



alp = {{{AFr' - AFf) (113) 



((..) denotes the average over the distributions Pf,Pr) respect all possible functions g^{W) 
shows [120^ H5] that the optimal solution is given by. 



with fi = AF. The same result has been obtained by Pande and coworkers by using maximum 
likelihood methods |121| . In this case, one starts from a whole set of work data encompassing 
Np forward and Nji reversed values. One then defines the likelihood function of distributing 
all work values between the forward and reverse sets. Maximization of the likelihood leads to 
Bennett's acceptance ratio formula. To extract AF is then customary to plot the difference 
in the Ihs of (jlOSp . Zfi{fi) — zp{fi), as a function of fi by using (|109lll0p . The intersection 
with the line zpi{ii) — zp{fj) = fi gives the best estimate for AF. An example of this method 
is shown in Figure [TTl Recently, the maximum likelihood method has been generalized to 
predict free energy differences between more than two states jl22j . 



5 Path thermodynamics 
5.1 The general approach 

The JE (I40p indicates a way to recover free energy differences by measuring the work along 
all possible paths that start from an equilibrium state. Its mathematical form reminds a lot 
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Figure 11: (Left) Bias as a function of the number of pulls N for a two-states model. The 
inset shows the number of pulls as a function of the dissipated work required to recover the free- 
energy with an error within IksT. (Right) Function zr — zp for the data shown in Figure\§P 
at the two largest pulling speeds. Left figure taken from 1100[ {3/- Right figure taken from the 
supplementary material in Ref. jllOl^ . 



of the partition function in the canonical ensemble used to compute free energies in statistical 
mechanics. The formulae for the two cases read, 



c 



exp(--j 



^ ^(r) AF, 
2^exp( — ) = exp( — — ^ 



(partition function) 



(Jarzynski equality) 



(115) 
(116) 



where F is the equilibrium free energy of the system at temperature T. Throughout this 
section we take /cs = 1. In the canonical ensemble the entropy S{E) is equal to the logarithm 
of the density of states with a given energy E. That density is proportional to the number of 
configurations with energy equal to E. Therefore (I115p becomes, 



exp 



(-^)=Eexp(-^)=Eexp(5(^) 



E 
T 



^exp 

E 



<^{E) 



(117) 



where ^{E) = E — TS is the free-energy functional. In the large volume limit the sum in (|117p 
is dominated by the value E = E^"^ where F{E) is minimum. The value E'^'^ corresponds to 
the equilibrium energy of the system and $(i?'^^) is the equilibrium free energy. The following 
chain of relations hold. 



F = $(E"<^) 



(dHEy. 

V dE ) E=E<"i 







( dS{E) ^ 

V dE ) E=E<='i 



1 

T 



(118) 



The equilibrium energy E'^'^ is different from the most probable energy, E^^ , defined by 
S'{E = S™P) = 0. is the average energy we would find if we were to randomly select 

configurations all with identical a priori probability. The equilibrium energy, rather than the 
most probable energy, is the thermodynamic energy for a system in thermal equilibrium. 
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Proceeding in an similar way for the JE we can define the path entropy S{W) as the 
logarithm of the density of paths with work equal to W, P{W), 

P{W) = exp{S{W)) . (119) 

We can rewrite (|116p in the following way, 



r 



T 



J dWexp[s{W) -y)= J dWexp[-^^) (120) 

where $(VF) = W — TS{W) is the path free energy. In the large volume limit the sum in 
(jl20p is dominated by the work value, , where <l>(Ty) is minimum. Note that the value 
plays the role of the equilibrium energy in the canonical case (IllSp . From ()120|] the path free 
energy $(VF^) is equal to the free energy difference AF. The following chain of relations hold, 

AF = <^{W^) = W^ -TS{W^) (121) 

V dW Jw=wi V dW Jw=wt T ^ ' 

At the same time, W'^ is different from the most probable work, defined as the work 

value at which S{W) is maximum, 

(^-^\ =0^f^^l =1 (123) 

The role of Vl^°'P,W^^ in the case of the JE (fUSl) and E^p,^^"^ in the partition function 
case (|116p appear exchanged. W"^^ is the work value typically observed upon repetition of the 
same experiment a large number of times. In contrast, in the partition function case (I115p . the 
typical energy is E'^^ rather than £^™p. In addition, is not the typical work but the work 
that must be sampled along paths in order to be able to extract the free energy difference using 
the JE. As we have already emphasized, as the system size increases, less and less paths can 
sample the region of work values around . Therefore, although both formalisms (partition 
function and JE) are mathematically similar, the physical meaning of the quantities and 
E'^^ is different. In the large volume limit E^^ is almost always observed whereas is almost 
never observed. 

In general, from the path entropy we can also define a path temperature, T{W), 

= X{W) = -J^ > fiW^) = T , (124) 

dW ^ ' T{W) ^ J ' V ^ 

where X{W) is a Lagrange multiplier that transforms the path entropy S{W) into the path 
free energy ^(W), equation (jl2ip . The mathematical relations between the new quantities 
VFt and W^p can be graphically represented for a given path entropy S{W). This is shown 
in Figure [12j 

The path thermodynamics formalism allows us to extract some general conclusions on the 
relation between and H^™p. Let us consider the CFT (j4ip . In terms of the path entropies 
for the forward and reverse processes, Sf{W) and SjiiW), (jlT]) can be written as, 

Sf{W) - Sni-W) = ^^^^ - {SpYiW) + (Snn-W) = i (125) 
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Figure 12: Upper panel: Path entropy s{w). Middle panel: Path free- energy ^{w) = w—Ts{w). 
Lower panel: Lagrange multiplier X{w) equal to the inverse of the path temperature 1/T{w). 
t(;™P is the most probable work value given by s'{w"^^) = A(?i;™P) = or ^'{w'^^) = 1; is 
the value of the work that has to be sampled to recover free energies from nonequilibrium work 
values using the JE. This is given by s'{w^) = 1/T or ^'{w^) = 0; Wj:ev,Wdi^ are the reversible 
and average dissipated work respectively. Figure taken from 1116]. 

where we used the definition (jllOp and later derived with respect to W . By inserting W = Wp 
and — VFJj in the rhs of (I125p and using (I122|123p we obtain the following chain of relations, 

{Sf)'{wI) + {Sr)'{-WI) = i ^ {Sr)'{-WI,) =Q^wl = -H^^P (126) 
{Sf)'{-W],) + {Sr)'{w],) = i - {SF)'{-Wi) = Q^wi = -W^^ . (127) 

The rightmost equalities in (|126ll27p imply that the most probable work along the forward 
(reverse) process is equal to the work value (W'^) that must be sampled, in a finite number 
of experiments, along the reverse (forward) process for the JE to be satisfied. This result has 
been already discussed in Sec. I4.2.3t the process that dissipates most between the forward 
and the reverse is the one that samples more efficiently the region of values in the vicinity of 
W'^ . This conclusion that may look counterintuitive can be rationalized by noting that larger 
dissipation implies also larger fluctuations and therefore more chances to get rare paths that 
sample the vicinity of W'^ . The symmetries (jl26ll27p were originally discussed in |116j and 
analyzed in detail for the case of the gas contained in a piston [11 7j . 

We close this section by analyzing the case where the work distribution is Gaussian. The 
Gaussian case describes the linear response regime usually (but not necessarily) characterized 
by small deviations from equilibrium. Let us consider the following distribution, 

P{W) = {2^al,)-\ exp(-^^^l— (128) 




42 



where the average value of the work, {W), is just equal to the most probable value The 
path entropy is given by S{W) = —(M/^ — H^™p)^/(2(T^) + constant, so (I123p is satisfied. From 
(HM]) we get, 



leading to the final result Wj^^^ = —W^^^ = — (Wdiss)- Therefore, in order to recover the 
free energy using the JE, paths with negative dissipated work and of magnitude equal to the 
average dissipated work must be sampled. Sometimes the paths with negative dissipated work 
are referred as transient violations of the second law. This name has raised strong objections 
among some rows of physicists. Of course the second law remains inviolate. The name just 
stresses the fact that paths with negative dissipated work must be experimentally accessible 
to efficiently recover free energy differences. Note that, for the specific Gaussian case, we 

2 

get (Wdiss) = "2^" ^'^d therefore the fluctuation-dissipation parameter R (H2|) is equal to 1 as 
expected for systems close to equilibrium. The result R = 1 has been shown to be equivalent 
to the validity of the fluctuation-dissipation theorem |95) . 

5.2 Computation of the work/heat distribution 

The JE and the GET describe relations between work distributions measured in NETS. How- 
ever, they do not imply a specific form for the work distribution. In small systems fluctuations 
of the work relative to the average work are large so work distributions can strongly deviate 
from Gaussian distributions and be highly non trivial. In contrast, as the system size increases, 
deviations of the work respect to the average value start to become rare and exponentially 
suppressed with the size of the system. To better characterize the pattern of nonequilibrium 
fluctuations it seems important to explore analytical methods that allow us to compute, at least 
approximately, the shape of the energy distributions (e.g. heat or work) along nonequilibrium 
processes. Of course there is always the possibility to carry out exact calculations in specific 
solvable cases. In general, however, the exact computation of the work distribution turns out 
to be a very difficult mathematical problem (solvable examples are [123^ 1124^ 11251 1126^ 1127) ) 
that is related to the evaluation of large deviation functions (Sec 15. 3p . This problem has 
traditionally received a lot of attention by mathematicians and we foresee it may become a 
central area of research in statistical physics in the next years. 

5.2.1 An instructive example 

To put the problem in proper perspective let us consider an instructive example: an individual 
magnetic dipole of moment /i subject to a magnetic field H and embedded in a thermal bath. 
The dipole can switch between the up and down configurations, it/i. The transition rates 




(129) 



From (fT22]l and (fT29ll we get = AF - {a^/2T). Therefore, 




(130) 



(131) 
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between the up and down orientations are of the Kramers type \128\ 1129] 




(132) 



(133) 



with ko = k^^p{H) + kdovrniH) independent of H. The rates (I132|133p satisfy detailed balance 



with P^i(+(— )/Li) = exp(— (+)/ui?/r)/Z where Z = 2cosh(^i7/T) is the equilibrium partition 
function. In this system there are just two possible configurations, C = —fi, fi. We consider a 
nonequilibrium protocol where the control parameter H is varied as a function of time, H{t). 
The dynamics of the dipole is a continuous time Markov process, and a path is specified by 
the time-sequence F = {fi{t)}. 

Let us consider the following protocol: the dipole starts in the down state —fi at if = —Hq. 
The field is then ramped from —Hq to +Hq at a constant speed r = H, so H(t) = rt 
(Figure [T3lA) . The protocol lasts for a time tmax = '^Hq jr and the field stops changing when 
it has reached the value Hq. The free energy difference between the initial and final state 
is because the free energy is an even function of H. To ensure that the dipole initially 
points down and that this is an equilibrium state, we take the limit oo but we keep the 

ramping speed r finite. In this way we generate paths that start at H = —oo at t = —oo and 
end up at = oo at t = oo. We can now envision all possible paths followed by the dipole. 
The up configuration is statistically preferred for H > whereas the down configuration is 
preferred for H < 0. Therefore, in a typical path the dipole will stay in the down state until 
the field is reversed. At some point, after the field changes sign, the dipole will switch from 
the down to the up state and remain in the up state for the rest of the protocol. In average 
there will be always a time lag between the time at which the field changes sign and the time 
at which the dipole reverses orientation. In another paths the dipole will reverse orientation 
before the field changes sign, i.e. when H < 0. These sort of paths become more and more 
rare as the ramping speed increases. Finally, in the most general case, the dipole can reverse 
orientation more than once. The dipole will always start in the down orientation and end in 
the up orientation with multiple transitions occurring along the path. 

The work along a given path is given by (I37p . 



Note that because EnifJ-) = E-H{—fi) then AE = and Q{T) = 1^(F) (i.e. the first law, 
AE = W — Q)) so heat and work distributions are identical in this example. Moreover, due to 
the time reversal symmetry of the ramping protocol, the work distribution P{W) is identical 
along the forward and reverse process. Therefore we expect that the JE (|4U|) and the CFT 
(HH) are both satisfied with AF = 0, 



m 




(134) 




(135) 




(136) 
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Figure 13: (Left) Ramping protocol. The ramping speed is defined by r = 2HQ/t where t is 
the duration of the ramp. (Right) Three examples of paths where the down dipole reverses 
orientation at different values of the field, H* . 

The exact computation of P{W) in this simple one-dipole model is already a very arduous task 
that, to my knowledge, has not yet been exactly solved. We can however consider a limiting 
case and try to elucidate the properties of the work (heat) distribution. Here we consider the 
limit of large ramping speed r where the dipole executes just one transition from the down 
to the up orientation. A few of these paths are depicted in Figure [T3B. This is also called a 
first order Markov process because it only includes transitions that occur along one direction 
(from down to up). In this reduced and oversimplified description a path is fully specified by 
the value of the field H* at which the dipole reverses orientation. The work along one of these 
paths is given by, 

[Ho 

W{T = H*) = - lim / dHii{H) = {{H* + Hq) - (Hq - H*))fi = 2^iH* (137) 

According to the second law, {W) = {Q) > 0, which implies that the average switching field is 
positive, {H*) > (as expected due to the time lag between the reversal of the field and the 
reversal of the dipole). The work distribution is just given by the switching field distribution 
p{H*). This is a quantity easy to compute. The probability that the dipole is in the down 
state at field H satisfies a master equation that only includes the death process, 



dpdownjH) ^ KpjH) 

dH ~ r 
This equation can be exactly solved. 



Pdo^H) . (138) 



Pdown 

{H) = exp(^-^J dHKpiH)) (139) 

where we have inserted the initial condition ^'down(~cxD) = 1. The integral in the exponent 
can be easily evaluated using (I132|133p . We get, 

Pdown(^) = (l + exp(^))"^ (140) 
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The switching field probabiUty distribution p{H*) is given by p{H*) = —(pdownYiH*). From 
(fT371) we get, 

P(^,.A(n.exp(^!:))--^=?<|l (141) 



4/xr V T / cosh(22'y 

and from this result we obtain the path entropy, 

S{W) = log{P{W)) = log(exp(^) + l) + ^ - iog(cosh(^)) + constant (142) 

It is important to stress that ()141|) does not satisfy ()136p except in the limit r — > oo where this 
approximation becomes exact. We now compute in the large r limit. We obtain to 

the leading order, 

^'(Vr'^P) =0^ VF'^P = riog(|^) + 0(-) (143) 

kqT r 

S'{W^) = 1 ^ lyt = -riog(|^) + O(-) (144) 

so the symmetry ()126p (or ()127p ) is satisfied to the leading order (yet it can be shown how 
the 1/r corrections appearing in VI^™p,1^^ (|143ll44p are different). We can also compute the 
leading behavior of the fluctuation-dissipation parameter R ()42p by observing that the average 



work (W) is asymptotically equal to the most probable work. The variance of the work, a'y^r, 
is found by expanding S{W) around 1^™p, 

S{W) = 5(T^"P) + ^ - Vr"P)2 + (higher order terms) (145) 

= - g.(^inp) (146) 

A simple computation shows that a^r = 2T and therefore, 

so R decays logarithmically to zero. The logarithmic increase of the average work with the 
ramping speed ()143p is just a consequence of the logarithmic increase of the average value 
of the switching field (H*) with the ramping speed. This result has been also predicted for 
the dependence of the average breakage force of molecular bonds in single molecule pulling 
experiments. This phenomenology, related to the technique commonly knownn as dynamic 
force spectroscopy, allows to explore free energy landscapes by varying the pulling speed over 
several orders of magnitude |13m I131j . 



5.2.2 A mean-field approach 

We now focus our attention in describing an analytical method useful to compute work distri- 
butions, P{W), in mean-field systems. The method has been introduced in |116j and developed 
in full generality by A. Imparato and L. Peliti [132|, 1133] . This section is a bit technical. The 
reader not interested in the details can just skip this section and go to Sec 15.31 
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The idea behind the method is the fohowing. We express the probabihty distribution 
P{W) as a sum over ah paths that start from a given initial state. This sum results in a path 
integral that can be approximated by its dominant solution or classical path in the large N 
limit, being the number of particles. The present approach exploits the fact that, as soon as 
N becomes moderately large, the contribution to the path integral is very well approximated 
by the classical path. In addition, the classical path exactly satisfies the FT. Here we limit 
ourselves to show in a very sketchy way how the method applies to solve the specific example 
shown in Sec. I5.2.1[ A detailed and more complete derivation of the method can be found in 
[I16l[l33] 

We come back to the original model (|132ll33p and include all possible paths where the 
dipole reverses orientation more than once. The problem now gets too complicated so what we 
do is to modify the original model by considering an ensemble of non-interacting identical 
dipoles. A configuration in the system is specified by the A^-component vector C = {/2 = 
{lJ'i)i<i<N} with fii = ±fi the two possible orientations of each dipole. A path is specified by 
the time-sequence T = {/3(s); < s < t}. The energy of the system is given by, 

N 

E{C) = -hM{C) = -hY,f^i (148) 

i=l 

where M = J2i Mi is total magnetisation. The equilibrium free energy is -F = — log(2 cosh.(fj,H /T)) 
and the kinetic rules are the same as given in (I132|133p and are identical for each dipole. The 
work along a given path is given by (I135p . 

W(T) = - t dsH{s)M{s) (149) 
Jo 

so the work probability distribution is given by the path integral, 

P{W) = P{V)8{W{V) -W)= J V[fl]6(w + dsH{s)M{s)') (150) 

where we have to integrate over all paths where fl starts at time in a given equilibrium state 
up to a final time t. To solve pSOh we use the integral representation of the delta function, 

/oo 
(iAexp(-iAx) . (151) 
-oo 

We also insert the following factor, 

1 = / »i exp( A I _ 1. Y, ,.(.))) (152) 



i=l 



where At is the discretization time-step and we have introduced new scalar fields j{s),m{s). 
After some manipulations one gets a closed expression for the work distribution P{w) {w = 
W/N is the work per dipole). We quote the final result |116| . 

P(w) = N I dXD[-i]V[m] exp(Afa(u;, A, 7, m)) (153) 
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where M is a normalization constant and a represents an action given by, 

a{w, X, 7, m) = X(^W + dsH{s)M{s)') + (154) 
^ ds(m{s){2^{s) + c{s)) + d{s)) + log(exp(7(0))A;,p(F,) + exp{-j{0)kdo^,,{H,)) (155) 
with, 

tdown(^(s))(exp(-27(s)) - l) - A:,p(F(s))(exp(27(s)) - l) (156) 



cis 



dis) = A;down(^(s)) (exp(-27(s)) - l) + KpiH{s)) (exp(27(s)) - l) (157) 

where the rates k-ap , A^down are given in ()132|133p and we have assumed an initial equilibrium 
state at the initial value of the field, H{0) = Hi. Equation (|155p has to be solved together 
with the boundary conditions, 

7(t) = ; m(0) = tanh(7(0) + ^) . (158) 

Note that these boundary conditions break causality. The function 7 has the boundary at the 
final time t whereas m has the boundary at the initial time 0. Causality is broken because by 
imposing a fixed value of the work w along the paths we are constraining the time evolution 
of the system. 

To compute P^w) we take the large volume limit ^ 00 in (|153|) . For a given value of w 
the probability distribution is given by, 

P{w) oc exp(A^s(ti;)) = e^p(^Na{w, X{w),ju){s),'mu){s)^ (159) 

where s is the path entropy (jll9p and the functions X{w),^w{s),mw{s) are solutions of the 
saddle point equations, 



Sa 

5a 



^ = w + fif m^{s)H{s)ds = (160) 
oX Jo 

. , . - mwis) + mwis){kup{s) + A;down(s)) - 
5-f{s) 

(fcup(s) - A;down(s)) + mw{s)dw{s) + c^(s) = (161) 
jUs) + Kw)t^H{s) + lc^{s) = . (162) 



5m{s) ' ' "^-^^^"^"^ ' 2 

These equations must be solved together with the boundary conditions (I158p . Note that we 
use the subindex (or the argument) w in all fields (A,m,7) to emphasize that there exists 
a solution of these fields for each value of the work w. From the entropy s in ()159p we can 
evaluate the path free energy, the path temperature and the values w'^p,w^ introduced in 
Sec. 15.11 We enumerate the different results: 

• The path entropy s{w). By inserting ()162p into ()155p we get, 

s{w) = X{w)w + ^ dy,{s)ds + log(exp{j{0))knp{Hi) + exp{--y{0))kdown{Hi)J . (163) 
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From the stationary conditions (jl60ll61ll62p the path entropy (jl59p satisfies, 

s'(w) = ^^^^^ = Mw,X{w),jw{s),m^{s)) ^ ^^^^ ^ ^^g^^ 
dw dw 

The most probable work can be determined by finding the extremum of the path entropy 

s{w), 

s'iw"'^) = A(u;°>P) = 0. (165) 

where we used (I123p . The saddle point equations (I160|161|162p give 7^mp(s) = c^mp(s) = 
d^„mp(s) = and, 

m^mp(s) = -m^mp(s)(A;up(s) + A;down(s)) + (^up(s) - ^down(s)) • (166) 

which is the solution of the master equation for the magnetisation. The stationary 
solution of this equation gives the equilibrium solution m'^'^{s) = tanh{fiH (s) /T) corre- 
sponding to a quasistationary or reversible process. 

• The path free energy. The path free energy / = <I>/A^ (I12ip is given by, 

AF T /■* 

/t = /(y;t) = — = w,e. = w^- Ts{w^) = - d^t {s)ds (167) 
iV Z Jo 

where w^ is given by, 

s'iw^) = X{w^) = ^ (168) 
and the path temperature (|124p satisfies the identity, 

n«^) = 3^ ; f{w^)=T (169) 

This set of equations can be numerically solved. Figure [TH shows some of the results. 
5.3 Large deviation functions and tails 

A large deviation function P{x) of a function Pl{x) is defined if the following limit exists, 

P(x) = lim ilog(Pi(^)) . (170) 

From this point of view the distribution of the entropy production in NESS, P{a) (|55p . where 
a = Sp/{Sp) and the work distribution, P{W) ()159p . define large deviation functions. In 
the first case, limt_>oo /i(a) is the large deviation function (e.g. (f5^ ). the average entropy 
production (Sp) being the equivalent of L in ()170p . In the second case the path entropy 
s{w) = S{W)/N (|119ll59p is a large deviation function, the size being the equivalent of L 
in ()170p . Large deviation functions are interesting for several reasons, 

• Nonequilibrium theory extensions. By knowing the large deviation function of an 
observable in a nonequilibrium system (e.g. the velocity or position density) we can 
characterize the probability of macroscopic fluctuations. For example, by knowing the 
function s{w) we can determine the probability of macroscopic work fluctuations 5W oc 
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Figure 14: Various results for the mean-field solution U6(M16U162\) of a dipole in field that 
is ramped from Hi = to Hf = 1. (A) Fields mx{s) and 7a (s) at the ramping speed r = 1. 
Curves correspond to different values of X (X = —5, —2, —1, —0.5, —0.2, 0., 0.2, 0.5, 1, 2, 5 from 
top to bottom in the upper and lower panel). The dashed line in m\{s) is the equilibrium 
solution meq{H) = tanh(//) corresponding to the reversible process r ^ 0. (B) Magnetization 
m\{s) for the most probable path A = 0. The dashed line corresponds to the reversible trajec- 
tory, r ^ 0. (C) Lagrange multiplier A(wdis) for three ramping speeds. The intersection of the 
different curves with the dashed line A = gives (filled circles) whereas the intersection 
with A = — 1 gives (filled squares). The intersection of all three curves around X = 0.5 is 
only accidental (looking at a larger resolution such crossing is not seen). (D) Path entropy 
and free energy corresponding to the solutions shown in (B,C) (larger speeds correspond to 
wider distributions). Path entropies are maximum and equal to zero at w™^ = — w^ew 
(filled circles) whereas path free-energies are minimum and equal to f^ = w^ev at 'U^dis (filled 
squares). Figure taken from fUEI- 
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A^, where N is the size of the system. Large deviations (for example, in work) may depend 
on the particular details (e.g. the rules) of the nonequilibrium dynamics. In contrast, 
small deviations (i.e. SW oc \/iV) are usually insensitive to the microscopic details of the 
dynamics. Nonequilibrium systems are non-universal and often strongly dependent on 
the microscopic details of the system. In this regard, understanding large or macroscopic 
deviations may be a first step in establishing a general theory for nonequilibrium systems. 

• Spectrum of large deviations. There are few examples where large deviations can 
be analytically solved. Over the past years a large amount of work has been devoted 
to understand large deviations in some statistical models such as exclusion processes. 
General results include the additivity principle in spatially extended systems ^134^ 1135^ 
I136j and the existence of exponential tails in the distributions [137j . These general 
results and the spectrum of large deviations are partially determined by the validity 
of the FT (I27p which imposes a specific relation between the forward and the reverse 
work/heat distributions. For example, exponential tails in the work distribution P{W) 
(I119P correspond to a path-entropy 5(1^) that is linear in W. This is the most natural 
solution of the FT, see (jl25p . 

• Physical interpretation of large deviations. In small systems, large deviations are 
common and have to be considered as important as small deviations. This means that, 
in order to understand the nonequilibrium behavior of small systems, a full treatment 
of small and large deviations is necessary. The latter are described by the shape of the 
large deviation function. The physical interpretation of small and large deviations may 
be different. For example, if we think of the case of molecular motors, small deviations 
(respect to the average value) in the number of mechanochemical cycles may be respon- 
sible of the average speed of a molecular motor whereas large deviations may be relevant 
to understand why molecular motors operate so efficiently along the mechanochemical 
cycle. 

5.3.1 Work and heat tails 

Let us consider the case of a NETS that starts initially in equilibrium and is driven out of 
equilibrium by some external driving forces. As we have seen in p59p . log(P(rt))) = s{w) 
is a large deviation function. At the same time we could also consider the heat distribution 
P{Q) and evaluate its large deviation function -^log(P(Q)) = s{q) where q = Q/N. Do we 
expect s{q) and s{w) to be identical? Heat and work differ by a boundary term, the energy 
difference. Yet the energy difference is extensive with N , therefore boundary terms modify 
the large deviation function so we expect that s{q) and s{w) are different. An interesting 
example is the case of the bead in the harmonic trap discussed in Sec. 14.11 Whereas the work 
distribution measured along arbitrary time intervals is always a Gaussian, the heat distribution 
is characterized by a Gaussian distribution for small fluctuations 5Q = Q— < Q >(x y/t, plus 
exponential tails for large deviations 5Q oc t. The difference between the large deviation 
function for the heat and the work arises from a boundary term, the energy difference. Again, 
in the large t limit, the boundary term is important for large fluctuations when a = \Q\/{Q) > 
a* = 1 (I84p . Large deviation functions depend always on boundary terms that often cannot 
be neglected. 
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Let us come back now to the example of Sec. l5.2.T] where we considered work distributions 
in a system of non-interacting dipoles driven by an externally varying magnetic field. Again 
we will focus the discussion on the particular case where the initial value of the field is negative 
and large Hi = —Hq — > — oo , and the field is ramped at speed r until reaching the final value 
Hf = Hq ^ CO. In this case Q = W ioi individual paths so both large deviation functions 
s{q),s{w) are identical. In what follows we will use heat instead of work for the arguments 
of all functions. In addition sp = sr due to the time-reversal symmetry of the protocol. 
Exponential tails are indicated by a path-temperature T{q) (jl24p which is constant along a 
finite interval of heat values. 

In Sec. 15. 2.1] we have evaluated the path entropy s{q) (jl63p for an individual dipole {N = 1) 
in the approximation of a first order Markov process. The following result has been obtained 

mi, 

s{q) = ^°S(^^P(^) + ^) + 2T ~ + constant . (171) 

For l^l ^ oo we get, 

s('?-oo) = -|^ + o(exp(-|)) (172) 



2/ir V T 
Q_ 
T 

The linear dependence of s{q) on q leads to 



,(g^_oo) = | + o(exp(|)) (173) 



f(g^oo)=r- = -^ (174) 

fco 

f{q -oo) = T+ = T (175) 

(176) 

where we use the notation to stress the fact that these path temperatures are positive 

and negative respectively. Both path temperatures are constant and lead to exponential tails 
for positive and negative work values. Note that equation (jl25p reads, 

s{q) - s{-q) = I - (snq) + (.)'(-g) = ^^± + ± = ^ (177) 

which is satisfied by (jl74ll75p up to 1/r corrections. 

Another interesting limit is the quasistatic limit r ^ 0. Based on the numerical solution 
of the saddle point equations (|160|161|162p . it was suggested in [116] that T{q) converged to 
a constant value over a finite range of work values. Figure [TSlA shows the results obtained for 
the heat distributions whereas the path temperature is shown in Figure [TSB. A more detailed 
analysis [133j has shown that a plateau is never fully reached for a finite interval of heat values 
when r — > 0. The presence of a plateau has been interpreted as the occurrence of a first order 
phase transition in the path entropy s{q) |133j . 

There is an interesting analogy between the different type of work/heat fluctuations and 
the emission of light radiation by atoms in a cavity. Atoms can absorb and reemit photons 
following two different mechanisms. One type of radiative mechanism is called stimulated 
because it depends on the density of blackbody radiation in the cavity (directly related to 
the temperature of the cavity). The other radiative mechanism is called spontaneous and is 
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Figure 15: (A) Heat distributions (path entropy s{q) and path free energy f{q)) evaluated at 
four ramping speeds r = 0.1, 0.5, 1, 10 (from the most narrower to the most wider distributions). 
The dashed line in the left panel is y{q) = q/T (we take T = 1) and is tangent to s{q) at q^ 
(dots are shown for r = 10). The dashed line in the right panel corresponds to y{q) = q and is 
tangent to the function f{q) at the value q^^ (dots shown for r = W). (B) \{q) for the lowest 
speed r = 0.1. It shows a linear behavior for small values of q, X{q) = (l/o'g)('?™P — q) and 
two plateaus for q >> 1 and q « —1. The former contributes as a Gaussian component to 
the heat distribution describing the statistics of small deviations respect to the most probable 
value (stimulated sector). The latter gives rise to two exponential tails for the distribution 
describing the statistics of rare events (spontaneous sector). Figure adapted from 1116^ - 



independent on the density of radiation in the cavity (i.e. it does not depend on its tem- 
perature). The stimulated process contributes to the adsorption and emission of radiation 
by atoms. The spontaneous process only contributes to the emitted radiation. In general, 
the path entropy s{w) contains two sectors reminiscent of the stimulated and spontaneous 
processes in the blackbody radiation, 

• The FDT or stimulated sector. This sector is described by Gaussian work fluctua- 
tions (fnH|) leading to s{q) = -{q - g™P)2/(2a^) + constant. Therefore we get (fT^ . 

1 a - a™P 

Kq) = j— = -^-I- (178) 

T{q) (Tq 

which behaves linearly in q for small deviations around g™^. Note that T{q) satisfies 
(I177P and therefore 

leading to a fluctuation-dissipation parameter R = 1, a result equivalent to the validity 
of the fluctuation-dissipation theorem (FDT). This sector we call stimulated because 
work fluctuations (I179p depend directly on the temperature of the bath. 

• The large deviation or spontaneous sector. Under some conditions this sector is 
well reproduced by exponential tails in the work distribution describing large or macro- 



53 



scopic deviations. In this sector, 

1 1 _ 1 1 1 _ 1 

fiqj ~ fi-q) ~T^T+^T^~T ' 

The physical interpretation of T^,T^ is as foUows. Because T~ is negative then T~ 
describes fluctuations where net heat is released to the bath, whereas is positive 
and describes fluctuations where net heat is absorbed from the bath. Equation ()180|] 
imposes < \T~\ implying that large deviations also satisfy the second law: the 
average net amount of heat supplied to the bath (oc \T~\) is always larger that the 
average net heat absorbed from the bath (oc T+). In the previous example ()174ll75p . 

converges to the bath temperature whereas T diverges to — oo when r oo. 
We call this sector spontaneous because the energy fluctuations mainly depend on the 
nonequilibrium protocol (in the current example, such dependence is contained in the 
r-dependence of , (jl74p ). 

5.3.2 The bias as a large deviation function 

The bias defined in p03p is still another example of a large deviation function. Let us define 
the variable, 

X = 5]exp(-^) (181) 

1=1 

where Wi Wi — AF stands for the dissipated work. The free-energy estimate (|103p satisfies 
the relation, 

7?JE _ ^jr X 1 ^ 

x = exp( ) ; x = - = -^exp(-iy,) (182) 

i=l 

where is the total number of experiments. The N values Wi are extracted from a distribution 
P{W) that satisfies the relations, 

/oo roo 
P{W)dW = 1 ; (exp(-iy)) = / exp{-W)P{W)dW = 1 . (183) 
-oo J ~oo 

We follow the same procedure as in Sec. 14.2.21 and extract N different values Wi to obtain a 
single X using (|182p . By repeating this procedure a large number of times, M, we generate 
the probability distribution of x, that we will call Vn{x), in the limit M ^ oo. The bias ()104p 
is defined by, 

/oo 
log{x)VNix)dx . (184) 
-oo 

In the following we show that Vm{x) defines a large deviation function in the limit — > oo. 
We write, 

N , TV 



'PNix) = Jfl dWiPiWi)6[x - ^ E exp(-T^, 

i=l i=l 
rioo ^ N 

/ dfiexpLx - E exp(-M^O) n P{W^)dW, 



i=l i=l 
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= — / dfiexp(Nilx + Nlog{ dWP{W)exp{-jlexp{-W)))) = 

= ^ y" d/iexp(^iV5((/i,x)) exp(A^5(/i*,x)) (185) 

where in the second hne we used the integral representation of the delta function (|15ip : in 
the third line we separate the integrals and independently integrate the contribution of each 
variable W^; in the last line we apply the saddle point integration method to the function 
x) defined as, 

g{fi,x) = /ix + log( / exp(— /i exp(— ) (186) 

—CO ^ 

where fi* is equal to the absolute maximum of g{fi,x), 

fdg^Y = - X =« exp(-Ty) (187) 

with, 

S-oo exp(-VF) exp(-/i exp(-W^)) P{W)dW 

« .. »A= 7 ^ X • (188) 

/"ooexp(-/iexp(-iy))p(Ty)dW^ 

The function g{fi,x) evaluated at ft = ft* defines a large deviation function (jl70p . 

g^x) = g{fL^x),x) = lim^^log{VNi^)) = lim^^lo^^ ■ (189) 

Using (I189p we can write for the bias (I184p in the large N limit, 

/ dx log(x) exp ( Ng* (x) ) 

B{N) = -T ^ . (190) 

/ dx exp(Ng*{x) j 

The integrals in the numerator and denominator can be estimated by using the saddle point 
method again. By expanding g*{x) around the maximum contribution at x™^^ we get, up to 
second order, 

g*{x) = 5*(x--) + i(5*)"(x---)(x - x--^)2 . (191) 
To determine x^^^ we compute first, 

'dg{fi,x)\ /djl*{x)\ /dg{fL*,x)^ 



dfi Jfi=fi*{x)\ dx ' V dx 

= (^^)=A-(.) (102) 

where we have used (11861187]) . The value x^^^ satisfies, 

(g*)'(x'"^") = /i*(x"^^") = . (193) 

Inspection of ()187|188p shows that x™^^ = 1. The second term in the rhs of ()19ip is then 
given by, 

fa''"'-=" = '^')'<--'>= l-(exp(-2W^)> <1^^) 
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where (...) denotes the average over the distribution P{W) (jl83p . Using ()194p and inserting 
(fm]) into (fTon]) we finally obtain, 



For a Gaussian distribution we get, B{N) = T exp{a^ — 1) / {2N) . Equation (|195p was derived 
in |103j . For intermediate values of N (i.e. for values of N where B{N) > 1) other theoretical 
methods are necessary. 

6 Glassy dynamics 

Understanding glassy systems (see Sec. 12. ip is a major goal in modern condensed matter 
physics |138t \1'69\ \14U\ I141| . Glasses represent an intermediate state of matter sharing some 
properties of solids and liquids. Glasses are produced by fast cooling of a liquid when the 
crystallization transition is avoided and the liquid enters the metastable supercooled region. 
The relaxation of the glass to the supercooled state proceeds by reorganization of molecular 
clusters inside the liquid, a process that is thermally activated and strongly dependent on 
the temperature. The relaxation of the supercooled liquid is a nonequilibrium process that 
can be extremely slow leading to aging. The glass analogy is very fruitful to describe the 
nonequilibrium behavior of a large variety of systems in condensed matter physics, all them 
showing a related phenomenology. 

The nonequilibrium aging state (NEAS, see Sec lS.lh is a non-stationary state characterized 
by slow relaxation and a very low rate of energy dissipation to the surroundings. Aging 
systems fail to reach equilibrium unless one waits an exceedingly large amount of time. For 
this reason, the NEAS is very different from either the nonequilibrium transient state (NETS) 
or the nonequilibrium steady state (NESS). 

What do aging systems have in common with the nonequilibrium behavior of small sys- 
tems? Relaxation in aging systems is driven by fluctuations of a small number of molecules 
that relax by releasing a small amount of stress energy to the surroundings. These molecules 
are grouped into clusters often called cooperatively rearranging regions (CRRs). A few obser- 
vations support this interpretation, 

• Experimental facts. Traditionally, the glass transition has been studied with bulk 
methods such as calorimetry or light scattering. These measurements perform an aver- 
age over all mesoscopic regions in the sample, but are not suitable to follow the motion 
of individual clusters of a few nanometers in extension. The few direct evidences we 
have on aging as driven by the rearrangement of small regions comes from AFM mea- 
surements on glass surfaces, confocal microscopy of colloids and the direct observation 
of molecular motion (NMR resonance and photobleaching tests) [8]. More indirect evi- 
dence is obtained from the heterogeneous character of the dynamics, i.e. the presence of 
different regions in the system that show a great disparity of relaxation times |l42j . The 
observation of strong intermittent signals |143j in Nyquist noise measurements while the 
system ages has been interpreted as the result of CRRs, i.e. events corresponding to the 
rearrangement of molecular clusters. Finally, the direct measure of a correlation length 
in colloidal glasses hints at the existence of CRRs |144j . Future developments in this 




(195) 
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area are expected to come from developments in micromanipulation and nanotechnology 
applied to the direct experimental observation of molecular clusters. 

• Numerical facts. Numerical simulations are a very useful approach to examine our 
understanding of the NEAS [145j . Numerical simulations allow to measure correlation 
functions and other observables which are hardly accessible in experiments. Susceptibil- 
ities in glasses are usually defined in terms of four-point correlation functions (two-point 
in space and two-point in time) which give information about how spatially separated 
regions are correlated in time |146j . A characteristic quantity is the typical length of 
such regions. Numerical simulations of glasses show that the maximum length of spa- 
tially correlated regions is small, just a few nanometers in molecular glasses or a few 
radii in colloidal systems. Its growth in time is also exceedingly slow (logarithmic in 
time) suggesting that the correlation length is small for the experimentally accessible 
timescales. 

• Theoretical facts. There are several aspects that suggest that glassy dynamics must 
be understood as a result of the relaxation of CRRs. Important advances in the under- 
standing of glass phenomena come from spin glass theory |147( 1148] . Historically, this 
theory was proposed to study disordered magnetic alloys which show nonequilibrium 
phenomena (e.g. aging) below the spin-glass transition temperature. However, it has 
been shown later how spin glass theory provides a consistent picture of the NEAS in 
structural glass models that do not explicitly contain quenched disorder in the Hamil- 
tonian [149^ 1150^ 1151^ I152j . Most of the progress in this area comes from the study of 
mean-field models, i.e. systems with long-range interactions. The success of mean-field 
theory to qualitatively reproduce most of the observed phenomenology in glasses suggests 
that NEAS are determined by the relaxation of mean- field like regions, maybe the largest 
CRRs in the system. Based on this analogy several mean-field based phenomenological 
approaches have been proposed [1531 1154] 11551 flSG} I157j . 

In the next sections I briefly discuss some of the theoretical concepts important to under- 
stand the glass state and the nonequilibrium aging dynamics. 

6.1 A phenomenological model 

To better understand why CRRs are predominantly small we introduce here a simple phe- 
nomenological aging model inspired by mean- field theory [154j . The model consists of a set 
of regions or domains of different sizes s. A region of size s is just a molecular cluster (col- 
loidal cluster), containing s molecules (or s colloidal particles). The system is prepared in an 
initial high energy configuration where spatially localized regions in the system contain some 
stress energy. That energy can be irreversibly released to the environment if a cooperative 
rearrangement of that region takes place. The release occurs when some correlated structures 
are built inside the region by a cooperative or anchorage mechanism. Anchorage occurs when 
all s molecules in that region move to collectively find a transition state that gives access 
to the release pathway, i.e. a path in configurational space that activates the rearrangement 
process. Because the cooperative process involves s particles, the characteristic time to anchor 
the transition state is given by. 




(196) 
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where r* = Toexp(B /T) is the activated time required to anchor one molecule, tq is a mi- 
croscopic time and B is the activation barrier which is equal to the energy of the transition 
state. How CRRs exchange energy with the environment? Once relaxation starts, regions of 
all sizes contain some amount of stress energy ready to be released to the environment in the 
form of heat. The first time a given region rearranges it typically releases an amount of heat 
Q that does not scale with the size of the region. After the first rearrangement has taken place 
the region immediately equilibrates with its environment. Subsequent rearrangement events 
in that same region do not release more stress energy to the environment. These regions can 
either absorb or release heat from/to the environment as if they were thermally equilibrated 
with the bath, the net average heat exchanged with the environment being equal to 0. The 
release of the stored stress energy in the system proceeds in a hierarchical fashion. At a given 
age t (the time elapsed after relaxation starts, also called waiting time) only the CRRs of size 
s* have some stress energy Q available to be released to the environment. Smaller regions 
with s < s* already released their stress energy sometime in the past being now in thermal 
equilibrium with the environment. Larger regions with s > s* have not had yet enough time to 
release their stress energy. Only the CRRs with s in the vicinity of s* contribute to the overall 
relaxation of the glass toward the supercooled state. That size s* depends on the waiting time 
or time elapsed since the relaxation started. 

Let ns{t) be the number of CRRs of size s at time t. At a given time the system is made 
out of non-overlapping regions in the system that randomly rearrange according to (11960 . 
After a rearrangement occurs CRRs destabilize probably breaking up into smaller regions. 
In the simplest description we can assume that regions can just gain or loose one particle 
from the environment with respective (gain, loose) rates k§, k\ with k§ + kl. = kg- kg, the rate 
of rearrangement, is proportional to l/r^ where is given in ()196p . To further simplify the 
description we just take /cf = gkg, ki. = Iks with g + l = 1. Consequently, the balance equations 
involve the following steps: 

Vs^Vs_i+p ; Vs+p^Vg+i (197) 

with rates A:^ , /cf , where Vg denotes a region of size s and p a particle (an individual molecule 
or a colloidal particle) in the system. The balance equations for the occupation probabilities 
read (s > 2), 

= A:i+in,+i(t) + kl_^ns^i{t) - /c,n,(t) . (198) 

This set of equations must be solved together with mass conservation J2'^i sns{t) = const. 
The equations can be numerically solved for all parameters of the model. Particularly inter- 
esting results are found for g <^ I. Physically this means that, after rearranging, regions are 
more prone to loose molecules than to capture them, a reasonable assumption if a coopera- 
tive rearrangement leads to a destabilization of the region. A few remarkable results can be 
inferred from this simple model, 

• Time dependent correlation length. In Figure [TH] (left panel) we show the time 
evolution for ns{t). At any time it displays a well defined time-dependent cutoff value 
s*{t) above which ns{t) abruptly drops to zero. The distribution of the sizes of the CRRs 
scales like ns{t) = {l/s*)h{s/s*) where s* is a waiting time dependent cutoff size (data 
not shown) . The NEAS can be parametrized by either the waiting time or the size of the 
region s*{t). Relaxation to equilibrium is driven by the growth of s*{t) and its eventual 
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convergence to the stationary solution of (jl98p . The size s*{t) defines a characteristic 
growing correlation length, ^(t) = {s*{t))d^ where d is the dimensionality of the system. 
Because s*{t) grows logarithmically in time (see (11960 ) sizes as small as ~ 10 already 
require 10^^ iteration steps. Small CRRs govern the relaxation of the system even for 
exceedingly long times. 

• Logarithmic energy decay. The release of stress energy to the environment occurs 
when the regions of size s* rearrange for the first time. The advance of the front in ns{t) 
located at s = s* is the leading source of energy dissipation. Cooperative rearrangements 
of regions of size smaller than s* have already occurred several times in the past and do 
not yield a net thermal heat flow to the bath whereas regions of size larger than s* have 
not yet released their stress energy. The supercooled state is reached when the cutoff 
size s* saturates to the stationary solution of (|198p and the net energy flow between the 
glass and the bath vanishes. The rate of energy decay in the system is then given by the 
stress energy Q released by regions of size s*{t) times their number ns*{t) = h{l)/s*, 
divided by the activated time (|196p (equal to the waiting time t ~ exp(i?s*/r)), 

dE ^ QnAtl ^ Q_ 

dt t s*t ' ^ ' 

Because s*{t) ~ T\og{t) then the energy decays logarithmically with time, E{t) ~ 
l/log(i). 

• Aging. If we assume independent exponential relaxations for the CRRs we obtain the 
following expression for the two-times correlation function, 

C(t, t + t') = Y^ sTisit) exp{-t'/Ts) (200) 

S>1 

where t denotes the waiting time after the initiation of the relaxation and is given by 
(|196p . In the right panel of Figurell6lwe show the correlation function (|200p for different 
values of t (empty circles in the figure). Correlations ()200p are excellently fitted by a 
stretched exponential with a t dependent stretching exponent Ps, 



C(t,t + t') = Ct(t') = exp(-(^)^^^*^) . (201) 



In the right panel of Figure [16] we also show the best fits (continuous lines) . Correlation 
functions show simple aging and scale like t'/t with t = exp(s*/T) where s* is the 
waiting-time dependent cutoff size. 

Configurational entropy and effective temperature. An important concept in 
the glass literature that goes back to Adam and Gibbs in the Fifties [1581 1159| is the 
configurational entropy, also called complexity and denoted by Sc |145j . It is proportional 
to the logarithm of the number of cooperative regions with a given free-energy F, 0,(F) 

5e(F) = log(j7(F)) . (202) 

At a given time t after relaxation starts, the regions of size s* contain a characteristic 
free energy F* . Fluctuations in these regions lead to rearrangements that release a net 
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amount of heat to the environment (jl99p . Local detailed balance implies that, after a 
rearrangement takes place, new regions with free energies around F* are generated with 
identical probability. Therefore, 



W{F F') n{F') 



W{F' F) n{F) 



exp(5e(F') -cSe(F)) (203) 



where W{F — > F') is the rate to create a region of free energy F' after rearranging a 
region of free energy F. Note the similarity between ()203p and ([8]). If AF' = F' — F is 
much smaller than Sc{F) we can expand the difference in the configurational entropy in 
(1203]) and write, 

with the shorthand notation W{AF) = W{F F') and the time-dependent effective 
temperature refj(F*) defined as. 



T,s{F*) V dF 

In the present phenomenological model only regions that have not yet equilibrated (i.e. 
of size s > s*{t)) can release stress energy in the form of a net amount of heat to the 
surroundings. This means that only transitions with AF < contribute to the overall 
relaxation toward equilibrium. 

Therefore, the rate of energy dissipated by the system can be written as, 

dE 1 dxxWjx) 2T,s{F*) 

—— OC n = ■ (20d 

dt t J^^dxWix) t 

where we take 

^ ^^^21^) ^'''^ 
as the solution of (l20l) . Identifying (l206]) and (fTM]) we get, 

T,s{F*) = §; . (208) 

The time dependence of s* derived in ()199p shows that the effective temperature de- 
creases logarithmically in time. 



6.2 Nonequilibrium temperatures 

The concept of a nonequilibrium temperature has stimulated a lot the research in the area 
of glasses. This concept has been promoted by Cugliandolo and Kurchan in the study of 
mean-field models of spin glasses \1G0\ I161j that show violations of the fluctuation-dissipation 
theorem (FDT) in the NEAS. The main result in the theory is that two-time correlations 
C{t,tw) and responses R{t,tw) satisfy a a modified version of the FDT. It is customary to 
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Figure 16: 7i<j(t) (left panel) and C{t,t + t') (right panel) for different waiting times t = 
10"^^ — 10^^ for the numerical solution of with I = 8, g = 1 and T = 0.45. The relaxation 

time and the stretching exponent are very well fitted by Tt = 2.2t^'^^ , f3s{t) = 0.34 + 0.45t~'''°^ . 
Figure taken from I154]l - 

introduce the effective temperature tlirougli tlie ffuctuation-dissipation ratio (FDR) [162J de- 
fined as, 

dC{t,t^) 

Tesit..) = (209) 

in the hmit where t — t^ ^ tw In contrast, in the hmit t — tyj t^ local equilibrium holds 
and Tf,Q{tyj) = T. In general Tcfi{tw) > T, although there are exceptions to this rule and even 
negative effective temperatures have been found [163]. These predictions have been tested 
in many exactly solvable models and numerical simulations of glass formers [145j . In what 
follows we try to emphasize how the concept of the effective temperature Teg(tw) contributes 
to our understanding of nonequilibrium fluctuations in small systems. 

Particularly illuminating in this direction has been the study of mean-field spin glasses. 
These models have the advantage that can be analytically solved in the large volume limit. At 
the same time, numerical simulations allow to investigate finite-size effects in detail. Theoreti- 
cal calculations in mean-field spin glasses are usually carried out by first taking the infinite-size 
limit and later the long time limit. Due to the infinite range nature of the interactions this 
order of limits introduces pathologies in the dynamical solutions and excludes a large spectrum 
of fiuctuations that are relevant in real systems. The infinite-size limit in mean-field models, 
albeit physically dubious, is mathematically convenient. Because analytical computations for 
finite-size systems are not available we can resort to numerical simulations in order to under- 
stand the role of finite-size effects in the NEAS. A spin glass model that has been extensively 
studied is the random orthogonal model (ROM) |164j , a variant of the Sherrington-Kirkpatrick 
model |165| . known to reproduce the ideal mode coupling theory |166| . The model is defined 
in terms of the following energy function, 

n = -Y^ J^J(Jiaj (210) 
{hi) 

where the ai are Ising spin variables {a = ±1) and is a random N x N symmetric 
orthogonal matrix with zero diagonal elements. In the limit N ^ oo this model has the same 
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thermodynamic properties as the random-energy model of Derrida |167l I168j or the p-spin 
model |169j in the large p limit [170^ I171j . The ROM shows a dynamical transition at a 
characteristic temperature T^yn (that corresponds to the mode coupling temperature Tmct in 
mode coupling theories for the glass transition [172J). Below that temperature ergodicity is 
broken and the phase space splits up into disconnected regions that are separated by infinitely 
high energy barriers. For finite the dynamics is different and the dynamical transition 
is smeared out. The scenario is then much reminiscent of the phenomenological model we 
discussed in Sec. 16.11 Different sets of spins collectively relax in finite timescales, each one 
representing a CRR. There are two important and useful concepts in this regard, 

• The free energy landscape. An interesting approach to identify CRRs in glassy 
systems is the study of the topological properties of the potential energy landscape 
|173j . The slow dynamics observed in glassy systems in the NEAS is attributed to the 
presence of minima, maxima and saddles in the potential energy surface. Pathways 
connecting minima are often separated by large energy barriers that slow down the 
relaxation. Stillinger and Weber have proposed to identify phase space regions with 
the so-called inherent structures (IS) [174^ I175j . The inherent structure of a region in 
phase space is the configuration that can be reached by energy minimization starting 
from any configuration contained in the region. Inherent structures are used as labels 
for regions in phase space. Figure [T7] (left panel) shows a schematic representation of 
this concept. Figure [TT] (right panel) shows the relaxation of the energy of the inherent 
structure energy starting from a high energy initial nonequilibrium state [1761 11771 1178j . 
Inherent structures are a useful way to keep track of all cooperative rearrangements that 
occur during the aging process [17'9] . 




• FD plots. Numerical tests of the validity of the FDR (j2U9|) use fluctuation-dissipation 
plots (FD plots) to represent the integrated response as a function of the correlation. 
The integrated version of relation (j2U9|) is expressed in terms of the susceptibility, 



where we have approximated Tcs{t') by Tcs{tu]). By measuring the susceptibility and the 
correlation function for a fixed value of and plotting one respect to the other, the slope 
of the curve x respect to C gives the effective temperature. This result follows naturally 
from (j212p if we take C{t,t) time independent (which is the case for spin systems). If 
not, proper normalization of the susceptibility and correlations by C{t, t) is required and 
a similar result is obtained |180j . A numerical test of these relations in the ROM and 
in binary Lennard-Jones mixtures is shown in Figure [181 We stress that these results 
have been obtained in finite-size systems. As the system becomes larger the timescales 
required to see rearrangements events become prohibitively longer and the relaxation of 
the system toward equilibrium drastically slows down. 






By introducing (|211|) in (j2U9|) we obtain, 




(212) 
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Figure 17: (Left) Stillinger and Weber decomposition. Schematic picture showing regions or 
components in phase space that are labeled by a given IS. (Right) Relaxation in the ROM. Panel 
(a): Equilibrium average eis a function of temperature. The arrows indicate the construction 
of the effective temperature T'efj(t^) 11209]) . Panel (b): Average inherent structure energy as 
function of time for the initial equilibrium temperature Ti = 3.0 and final quench temperatures 
Tf = 0.1, 0.2, 0.3 and 0.4. The average is over 300 initial configurations. The system size is 
N = 300. Left figure taken from 1179^ . Right figure taken from 1177\ \178 ^ 




Figure 18: (Left panel) Integrated response function as a function of IS correlation function, 
i.e, the correlation between different IS configurations, in the ROM. The dashed line has slope 
Tf^ = 5.0, where Tf is the final quench temperature, whereas the full lines are the prediction 
from m^) and F* = Eis{ty,): T^si^^'^) ^ 0.694, Tes{2^^) ^ 0.634 and Tes{2^^) ^ 0.608. 
The dot-dashed line has slope l/refj(t^) for t„ = 2^^ and is drawn for comparison. Figure 
taken from \1 71 1 (Right panel) Integrated response versus the dynamical structure factor for 
the binary mixture Lennard-Jones particles system in a quench from the initial temperature 
Ti = 0.8 to a final temperature Tf = 0.25 and two waiting times tyj = 1024 (square) and 
tw = 16384 (circle). Dashed lines have slope l/Tf while thick lines have slope 1/Tes_{tw)- 
Figure taken from 11181^ . 
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6.3 Intermittency 



Indirect evidence of nonequilibrium fluctuations due to CRRs in structural glasses has been 
obtained in Nyquist noise experiments by Ciliberto and coworkers. In these experiments a 
polycarbonate glass is placed between the plates of a condenser and quenched at temperatures 
below the glass transition temperature. Voltage fluctuations are then recorded as a function 
of time during the relaxation process and the effective temperature is measured, 



where Tl{Z{u!,tw)) is the real part of the impedance of the system and Sz{uJ,tw) is the noise 
spectrum of the impedance that can be measured from the voltage noise |143j . 

Experimental data shows a strong variation of the effective temperature with the waiting 
time by several orders of magnitude. The voltage signal is also intermittent with strong 
voltage spikes at random times. The distribution of the times between spikes follows a power 
law characteristic of trap models. These results point to the fact that the observed voltage 
spikes correspond to CRRs occurring in the polycarbonate sample. Finally, the probability 
distribution function (PDF) of the voltage signal strongly depends on the cooling rate in 
the glass suggesting that relaxational pathways in glasses are very sensitive to temperature 
changes. A related effect that goes under the name of Kovacs effect has been also observed in 
calorimetry experiments, numerical simulations and exactly solvable models \182\ 11831 1184j . 

A physical interpretation of the intermittency found in aging systems has been put forward 
based on exactly solvable models of glasses jl85 1ll86"lll87j . According to this, energy relaxation 
in glassy systems follows two different mechanisms (see Sec. I5.3.T|1 : stimulated relaxation and 
spontaneous relaxation. In the NEAS the system does not do work but exchanges heat with the 
environment. At difference with previous Sections here we adopt the following convention: Q > 
(Q < 0) denotes heat absorbed (released) by (from) the system from (to) the environment. 
Energy conservation then reads AE = Q + W where W is the worked done on the system. In 
the NEAS, W = and AE = Q, the energy released by the system is dissipated in the form 
of heat. In the phenomenological model put forward in Sec. 16.11 different CRRs can exchange 
(adsorb or release) heat to the environment. The regions that cooperatively rearrange for the 
first time release stress energy to the environment and contribute to the net energy dissipation 
of the glass. We call this mechanism the spontaneous relaxation. Regions that have already 
rearranged for the first time can absorb or release energy from/to the bath several times but 
do not contribute to the net heat exchanged between the system and the bath. We call this 
mechanism the stimulated relaxation. There are several aspects worth mentioning, 

• Heat distribution. The distribution of heat exchanges Q = E{tu,) — E{t) for the 
stimulated process is a Gaussian distribution with zero mean and finite variance. This 
process corresponds to the heat exchange distribution of the system in equilibrium at 
the quenching temperature. In contrast, in the spontaneous process a net amount of 
heat is released to the bath. Spontaneous heat arises from the fact that the system has 
been prepared in a nonequilibrium high-energy state. Let us consider a glass that has 
been quenched at a temperature T and has relaxed for an age t^. During aging, CRRs 
that release stress energy (in the form of heat Q < 0) to the environment satisfy the 




(213) 
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relation (|20i|l . 

P'^Q) ( Q \ 



(214) 



psp(_Q) ^\t,s{F*)) 



Therefore, as in the phenomenological model (j207p . we expect, 



P^P(Q) oc exp 



( 



Q 



) for Q < 



(215) 



2Teff(F*) 



Note that Tes{F*) depends on the age of the system through the value of the typical 
free-energy of the CRRs that release their stress energy at i^, F*{tw)- This relation has 
been numerically tested in the ROM (I210p by carrying out aging simulations at different 
temperatures and small sizes |185j (see the next item). 

• Numerical tests. How to measure the heat distribution ()215p in numerical simulations 
of NEAS? A useful procedure uses the concept of inherent structures and goes as follows. 
The heat exchanged during the time interval [tw,t] {t > tw) has to be averaged over 
many aging paths (ideally an infinite number of paths). Along each aging path many 
rearrangement events occur between tu, and t. Most of them are stimulated, a few of them 
are spontaneous. In fact, because the spontaneous process gets contribution only from 
those cooperative regions that rearrange for the first time, its PDF signal gets masked 
by the much larger one coming from the stimulated component where rearrangements 
events from a single region contribute more than once. To better disentangle both 
processes we measure, for a given aging path, the heat exchange corresponding to the 
first rearrangement event observed after t^. To identify a rearrangement event we keep 
track of the IS corresponding to the run time configuration. Following the IS is an 
indirect way of catching rearranging events due to CRRs. Only when the system changes 
IS we know that a cooperative rearrangement event has taken place. Rearrangement 
events take place at different times t after tw, therefore the heat distribution P^p(Q) 
is measured along an heterogeneous set of time intervals. The results for the heat 
distributions at various ages are shown in Figure [19l We notice the presence of 
two well defined contributions to the heat PDFs: A Gaussian central component plus 
additional exponential tails at large and negative values of Q. The Gaussian component 
corresponds to the stimulated process, however its mean is different from zero. The 
reason of this apparent discrepancy lies in the numerical procedure used to measure the 
heat PDF: the average stimulated heat is not equal to the net exchanged heat (which 
should be equal to 0) because different aging paths contribute to the heat exchange along 
different time intervals. The Gaussian component should be equal to the heat PDF for 
the system in thermal equilibrium at the same temperature and therefore independent 
of tuj . Indeed the variance of the Gaussian distribution is found to be independent of tyj 



• Spontaneous events release stress energy. One striking aspect of the spontaneous 
process is that, according to (I214p . the probability of heat absorption {Q > 0) should 
be much larger than the probability of heat release {Q < 0). However, this is not 
observed in the numerical results of Figure [19] where the exponential tail is restricted to 
the region Q < 0. Why spontaneous events are not observed for Q > 0? The reason is 
that spontaneous events can only release and not absorb energy from the environment, 
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Figure 19: Heat exchange PDFs for T = 0.3 (a), T = 0.2 (b), T = 0.1 (c). Circles are for 
tw = 2^° and asterisks for t^ = 2^^ . The continuous lines are Gaussian fits to the stimulated 
sector, the dashed lines are the exponential fits to the spontaneous sector. Figure taken from 

see (I215p . This is in the Hne of the argumentation put forward in Sec. 16.11 where it 
was said that the first time that cooperative regions release the stress energy, this gets 
irreversibly lost as heat released to the environment. As the number of stressed regions 
monotonically decreases as a function of time the weight of the heat exponential tails 
decreases with the age of the system as observed in Figure [THl The idea that only 
energy decreasing events contribute to the effective temperature ()215p makes it possible 
to define a time-dependent configurational entropy |188j . 

• Zero-temperature relaxation. This interpretation rationalizes the aging behavior 
found in exactly solvable entropy barrier models that relax to the ground state and 
show aging at zero temperature |189t llDOj . At T = 0, the stimulated process is sup- 
pressed (microscopic reversibility ^ does not hold), and (|204|) holds by replacing the 
free energy of a CRR by its energy, F = E. In such models a region corresponds to just a 
configuration in phase space and relaxation occurs through spontaneous rearrangements 
where configurations are visited only once. In entropic barrier models the effective tem- 
perature (|205|) still governs aging at T = 0. Because the energy is a monotonically 
decreasing quantity for all aging paths, (I204p does not strictly hold as W(Ai? > 0) = 0. 
Yet, the effective temperature obtained from (|204p has been shown to coincide with that 
derived from the FDR (fM [1861 [W]. 
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7 Conclusions and outlook 



We have presented a general overview of several topics related to the nonequilibrium behavior 
of small systems: from fluctuations in mesoscopic systems such as small beads in optical 
traps up to molecular machines and biomolecules. The main common theme being that, 
under appropriate conditions, physical systems exchange small amounts of energy with the 
environment leading to large fluctuations and strong deviations from the average behavior. 
We call such systems small because of their properties and behavior are markedly different 
from macroscopic systems. We started our tour by stressing the similarities between colloidal 
systems and molecular machines: intermittency and nonequilibrium behavior are common 
aspects there. We have then discussed fluctuation theorems (FTs) in detail, and focused 
our discussion in two well studied systems: the bead in a trap and single molecule force 
experiments. Experimental results in such systems show the presence of large tails in heat and 
work distributions in marked contrast to Gaussian distributions, characteristic of macroscopic 
systems. Such behavior can be rationalized by introducing a path formalism that quantifies 
work/heat distributions. Finally, we revised some of the main concepts in glassy dynamics 
where small energy fluctuations appear an essential underlying ingredient of the observed 
slow relaxation. Yet, we still lack a clear understanding of the right theory that unifies 
all phenomena and a clear and direct observation of the postulated small and cooperatively 
rearranging regions remains an experimental challenge. We envision three future lines of 
research, 

• Developments in FTs. FTs are simple results that provide a new view to better 
understand issues related to irreversibility and the second law of thermodynamics. The 
main assumption of FTs is microscopic reversibility or local equilibrium, an assumption 
that has received some criticism |19H 119!^ 1193) . Establishing limitations to the validity 
of FTs is the next task for the future. At present no experimental result contradicts any 
of the FTs, mainly because the underlying assumptions are respected in the experiments 
or because current experimental techniques are not accurate enough to detect systematic 
discrepancies. Under some experimental conditions, we might discover that microscopic 
reversibility breaks down and a more refined and fundamental description of the relevant 
degrees of freedom in the system becomes necessary. Validation of FTs under different 
and far from equilibrium conditions will be useful to test the main assumptions. 

• Large deviation functions. The presence of large tails can be investigated in statis- 
tical mechanics theories by exact analytical solutions of simple models, by introducing 
simplified theoretical approaches or even by designing smart and efficient algorithms. In 
all cases, we expect to obtain a good theoretical understanding of the relation between 
large deviations and nonequilibrium processes. Ultimately, this understanding can be 
very important in biological systems where nonequilibrium fluctuations and biological 
function may have gone hand by hand during biological evolution on earth over the past 
4.5 billion years. A very promising line of research in this area will be the study of molec- 
ular motors where the large efficiency observed at the level of a single mechanochemical 
cycle might be due to a very specific adaptation of the molecular structure of the en- 
zyme to its environment. This fact may have important implications at the level of 
single molecules and cellular structures. 
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• Glassy systems. We still need to have direct and clear experimental evidence of the 
existence of the cooperatively rearranging regions, responsible for most of the observed 
nonequilibrium relaxational properties in glasses. However, the direct observation of 
these regions will not be enough. It will be also necessary to have a clear idea of how to 
identify them in order to extract useful statistical information that can be interpreted 
in the framework of a predictive theory. Numerical simulations will be very helpful in 
this regard. If the concept of nonequilibrium temperature has to survive the time then 
it will be necessary also to provide accurate experimental measurements at the level of 
what we can now get from numerical simulations. 

Since the discovery of Brownian motion in 1827 by the biologist Robert Brown and the 
later development of the theory for the Brownian motion in 1905, science is witnessing an 
unprecedented convergence of physics toward biology. This was anticipated several decades 
ago by Erwin Schrodinger who, in his famous monography published in 1944 and entitled What 
is life? [194J, wrote when talking about the motion of a clock: "The true physical picture 
includes the possibility that even a regularly going clock should all at once invert its motion 
and, working backward, rewind its own spring -at the expense of the heat of the environment. 
The event is just 'still a little less likely' than a 'Brownian fit' of a clock without driving 
mechanism". Biological systems seem to have exploited thermal fluctuations to built new 
molecular designs and structures that efficiently operate out of equilibrium at the molecular 
and cellular level [1951 1196^ 1197^ I198j . The synergy between structure and function is most 
strong in living systems where nonequilibrium fluctuations are at the root of their amazing 
and rich behavior. 
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8 List of abbreviations 

GET Crooks fluctuation theorem 

JE Jarzynski equality 

EDT fluctuation-dissipation theorem 

EEC Eorce-extension curve 

ET Fluctuation theorem 

IS inherent structure 

PDE probability distribution function 
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